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ABSTRACT 

This  thesis  investigates  the  relationship  between 
Washington  based  Project  Managers  and  a  Test  and  Evaluation 
(TSE)  facility,  the  Pacific  Missile  Test  Center  (PMTC)  .  Five 
areas  of  the  relationship  are  examined.  First,  the  formal 
weapon  system  acquisition  process  as  prescribed  by  current 
directives  is  presented  as  an  overview.  Next,  the  Pacific 
Missile  Test  Center  organization  is  analyzed.  This  is 
followed  by  an  investigation  of  funding  techniques,  and 
specifically,  those  employed  by  PMTC.  The  Project  Managers* 
perception  of  their  participation  in  the  TSE  process  is 
presented  through  results  of  a  questionnaire.  Finally,  an 
objective  method  of  selecting  the  appropriate  TSE  facility 
for  a  project  based  on  technical  capability  is  developed. 
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I.  INTRODUCTION 

The  authors'  previous  educational  background  in 
Aeronautical  Engineering  combined  with  their  current 
education  in  Administrative  Sciences  stimulated  interest  in 
the  Weapons  System  Acquisition  Management  (WSAH)  field.  It 
was  hoped  that  thesis  research  in  this  area  would  benefit 
the  authors  in  future  billet  assignments  in  this  area.  In 
order  to  focus  on  a  specific  topic,  a  literature  search  of 
current  research  in  this  area  was  conducted.  The  search 
indicated  a  number  of  interesting  problems  involving  the 
Test  and  Evaluation  (r&E)  aspects  of  WSAM.  It  was 
considered  desirable  to  concentrate  on  the  focal  point  of 
T&E,  namely,  the  Washington  D.C.  arena.  However,  due  to 
lack  of  funding  support,  trips  to  the  Washington  area  were 
considered  impractical.  Consequently,  problems  involving 
T&E  activities  in  the  local  region  were  investigated. 
Contacts  were  made  at  both  the  Naval  Weapon  Center  (NWC) , 
China  Lake,  and  the  Pacific  Missile  Test  Center  (?MTC) , 
Point  Mugu. 

A  trip  was  made  to  point  Mugu  for  f amilarization  and 
problem  definition  purposes.  Personnel  at  Point  Mugu  were 
very  receptive  and  interested  in  cooperating  with  thesis 
research  involving  PMTC.  After  the  initial  trip,  it  was 
decided  to  concentrate  on  analyzing  the  T&E  facility/Project 
Manager  relationship.  Additional  trips  made  to  Point  Mugu 
focused  on  understanding  the  activity's  involvement  in  the 
T&E  process. 

The  next  five  chapters  represent  the  results  of  this 
research  with  the  first  four  chapters  concentrating  each  on 
a  separate  aspect  of  the  relationship.  First,  the  formal 
weapon  system  acquisition  process  (principly  the  T&E 
aspects)  as  prescribed  by  current  directives  is  presented 
for  purposes  of  providing  an  overview   of   how   the   process 
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should  work.  Next,  a  T&E  activity's  (PMTC)  organizational 
development  is  described  together  with  how  the  organization 
interacts  with  its  external  environment.  Presupposing  that 
"he  who  has  the  gold  rules",  the  financial  management  of  the 
facility  is  investigated.  This  is  followed  by  a  different 
viewpoint  of  the  relationship:  the  Project  Manager's 
perception  of  his  participation  in  the  T&E  process.  The 
Project  Manager's  opinions  were  ascertained  by  means  of  a 
questionnaire.  Results  of  the  questionnaire  provide  the 
data  for  Chapter  VI.  This  chapter  proposes  an  objective 
method  of  selecting  the  appropriate  T&E  facility  for  a 
particular  weapon  system's  T&E  requirements  based  on 
technical  capabilities. 
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II.  T.EST  AND  EVALUATION  WITHIN  THE  NAVY 

A.     •  ORGANIZATION  FOR  RDT&E 

Reference  1  provides  a  general  overview  of  the  Navy 
RDT&E  organization  and  process.  Much  of  the'  following 
information  has  been  extracted  from  it  in  order  to  briefly 
acquaint  the  reader  with  pertinent  background  information. 

In  the  National  Security  Act  of  1947  Congress  tasked 
the  Secretary  of  Defense  to  : 

"...eliminate  unnecessary  duplication  in  the  Department 
of  Defense  and  particularly  in  the  field  of  research  and 
engineering. . . " 

Later,  the  Defense  Reorganization  Act  of  1958 
established  the  Director  of  Defense  Research  and  Engineering 
(DDR&E)  as  the  highest  ranking  assistant  secretary  of 
defense.  The  primary  responsibilities  of  DDR&E  are  to  act  as 
principal  advisor  to  the  Secretary  of  Defense  (SECDEF)  on 
scientific  and  technical  matters,  supervise  research  and 
engineering,  and  direct  and  control  research  and  engineering 
activities.  To  fulfill  his  responsibilities  he  has  been 
delegated  authority  to  approve,  modify,  or  disapprove 
programs  and  projects  of  the  Military  Departments.  DODDTR 
5100.1  (Ref.  2)  established  the  Deputy  Director (Research  and 
Advanced  Technology)  (R&AT)  and  the  Deputy  Director (Test  and 
Evaluation)  (DDT&E)  as  advisors  to  the  Director. 

In  addition  to  the  above  DOD  organization,  the  Joint 
Chiefs  of  Staff  (JCS)  has  the  responsibility  of  advising 
SECDEF  on  matters  of  strategic  guidance,  overall  military 
requirements,  and  recommendations  of  new  weapon  systems. 
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Figure  1  depicts  the  Department  of  the  Navy 
Organization  for  RDT&E.  While  the  overall  responsibility  for 
policies  and  control  of  the  -  Department  of  the  Navy  rests 
with  the  Secretary  of  the  Navy.  Matters  related  to  research, 
development,  engineering,  test  and  evaluation  are  the 
specific  responsibility  of  the  Assistant  Secretary  of  the 
Navy  (R&D)  (ASN  (R&D) )  as  directed  by  SECN&VINST  5430.67  (Ref. 
3)  .  He  and  his  small  staff  act  as  the  principal  contact  for 
DDR&E  and  manage  the  "Research,  Development,  Test  and 
Evaluation,  Navy"  appropriations  account.  Due  to  his  limited 
staff  size  he  is  dependent  on  advisors  such  as  Director 
RDT&E  (OP-098) ,  the  Chief  of  Naval  Development  (CND) ,  Chief 
of  Naval  Research  (CNR)  ,  and  the  Mrector  of  Navy 
Laboratories  (DNL) . 

The  Office  of  the  Chief  of  Na»al  Operations  is 
primarily  concerned  with  defining  potentially  attainable 
capabilities,  appraising  the  military  worth  of  RDT&E 
advancements,  and  approving  projects  which  promise  the 
greatest  return  on  invested  resources.  These  policies  are 
specifically  implemented  by  DRDT&E  (OP-098)  who  acts  as  the 
focal  point  for  T&E  policy. 

The  "how"  aspects  of  Navy  RDT&E  have  been  delegated  to 
the  Chief  of  Naval  Material  (CNM) .  His  responsibilities 
include: 

1-   Translating   operational   requirements    into 
hardware  systems  and  technology  objectives. 

2.  Managing  the  technology  base. 

3.  Defining  science  and  technology  capabilities. 

4.  Developing  detailed  plans  for  RDT&E  projects. 
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5,  Overseeing  implementation  of  RDTSE  programs. 
i 

CNM  is  assisted  in  the  implementation  of  the   above   by 
the   Deputy   Chief   of  Naval  Material (Development)   (DCNM  (D) ) 
who  also  acts   as   Chief   of   Naval   Development   (CND)   and 
reports  directly  to  ASN  (R&D) . 

The  Naval  Material  Command  consists  of  NAVMAT 
designated  Project  Managers  (PMs)  plus  the  various  Systems 
Commands  as  depicted  in  Figure  2.  NAVMAT  designated  PMs  are 
utilized  for  projects  with  critical  interfaces  between 
Military  Services  and/or  Systems  Commands.  For  other 
projects,  one  of  the  Systems  Commands  is  placed  in  charge 
and  Projects  Managers  are  designated  within  it. 

NAVAIRINST  5451.80  (Ref.  4)  established  the  Project 
Management  Office (AIR)  within  NAVAIR  Headquarters  to  assist 
in  the  planning,  directing,  and  controlling  general 
management  aspects  of  specific  weapons  system  projects.  As 
such,  they  are  responsible  for  maintaining  the  NAVAIR 
designated  project  charters  (5430  instruction  series) , 
communicating  broad  policy  guidance  to  PMs  from  higher 
authority,  and  providing  other  assistance  to  PMs  as 
required. 

The  project  management  organization  is  structured  by 
purpose  and  thus  cuts  across  various  functional  organization 
lines.  PMs  are  generally  Navy  Captains  or  Flag  Officers 
available  for  at  least  a  three  year  tour.  OPNAVINST  3960.8 
(Ref.  5)  charges  the  PM  with  the  responsibility  for 
coordination  and  execution  of  a  T&E  program  responsive  to 
the  policies  expressed  in  SECNAVINST  5000. 1  (Ref.  6) . 
Specifically,  he  must  coordinate  a  test  program  with  the  CNO 
development  coordinator  suitable  to  answer  operational 
questions,  prepare  a  Test,  and  Evaluation  Master  Plan  (TEMP), 
arrange  for  the   performance   of   T&E   and   T&E   support   at 
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appropriate  laboratories,  contractor  facilities  or  T&E 
activities,  and  determine  the  need  for  assistance  of 
operating  forces. 

Laboratories  responsible  for  R&D  of  aircraft  and 
missile  systems  report  directly  to  the  Director  of 
Laboratory  Programs  (DLP)  in  the  staff  of  DCNK  (D)  .  The  DLP, 
also  acting  as  DNL,  is  the  principal  advisor  to  ASN(R&D)  in 
matters  of  the  RDT&E  field  complex. 

Test  and  Evaluation  of  aircraft  and  aircraft  weapon 
systems  is  generally  divided  into  Operational  Test  & 
Evaluation  (OT&E)  and  Development  Test  &  Evaluation  (DT&E) . 
OT&E  is  the  responsibility  of  the  VX  squadrons  of  the 
Operational  Test  and  Evaluation  Forces  (OPTEVFOPv)  which 
report  to  CNO  via  OP-983,  the  Test  and  Evaluation  Division 
of  OP-098.  DT&E  is  generally  under  the  cognizance  of  field 
activities  such  as  the  Naval  Air  Test  Center  and  Pacific 
Missile  Test  Center  which  report  to  NAVAIR.  The  coordination 
of  T&E  activity  within  NAVAIR  is  currently  delegated  to  the 
Test  and  Evaluation  Coordinator  (TECO)  who  is  double-hatted 
as  CO  NATC  as  directed  by  NAVAIRINST  5400. 27B  (Ref .  7) . 

Some  of  the  specific  tasks  of  TECO  are:  provide  to 
NAVAIR  recommendations  and  proposed  directives  concerning 
T&E  policy,  act  as  focal  point  for  T&E  matters,  assist  in 
preparation  of  and  evaluate  planning  documents  containing 
T&E  requirements,  provide-  comments  to  NAVAIR  on  adequacy  of 
T&E  plans,  review  and  evaluate  workload  assignment  to  field 
activities,  monitor  actual  T&E  accomplishment  in  order  to 
recommend  readjustment  of  available  resources,  provide 
planning  information  to  field  activities,  and  assist  NAVAIR 
PMs  in  the  coordination  with  COMOPTEVFOR. 

NAVAIR    NOTICE    5400   of   18   March,   1975   (Ref.   8) 
established  the   Assistant   Commander   for   Test   Evaluation 
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(AIR-06)  and  called  for  an  AIR-06  operational  status  by  1 
July  1975.  Ke  is  responsible  for  the  policy  direction, 
planning,  management  and  support  of  NAVAIRSYSCOM  T&E 
programs,  facilities,  ranges,  and  targets.  He  is  the  primary 
NAVMAT  contact  with  contractors  and  other  government 
agencies  for  matters  concerning  T&E  facilities,  ranges,  and 
targets  including  all  Navy  elements  of  the  Department  of 
Defense  Major  Ranges  and  Test  Facilities  Base.  Specific 
duties  include;  assisting  Project  Managers  in  preparing  for 
Defense  Systems  Acquisition  Review  Council  (DSARC)  reviews 
and  in  budget  preparation,  providing  life-cycle  management 
of  targets  and  T&E  support  systems,  providing  guidelines  for 
and  approving  mission  functions,  organization  and  staffing 
of  T&E  field  activities  and  ranges,  establishing  the  policy 
for  the  assignment  of  workload  to  the  T&E  field  activities, 
establishing  priorities  for  the  use  of  T&E  activities,  and 
maintaining  liaison  with  COMOPTEVFOR  to  assure  a  coordinated 
DT&E  and  OT&E  of  NAVAIR  weapon  systems  and  equipment.  The 
presently  anticipated  organization  within  AIR-06  calls  for 
an  AIR-06T  (double-hatted  as  CO  NATC)  who  would  be 
responsible  for  T&E  efforts  and  AIR-06R  (double-hatted  as  CO 
PMTC)  who  would  be  responsible  for  all  ranges  and  targets. 

B.    PLANNING  PROCESS 

The  planning  system  leading  to  OPN AV-supported  RDT&E, 
weapon  systems  has  recently  been  revised  from  the  old  system 
in  effect  since  1961.  This  paper  will  address  the  new 
system;  however,  projects  approved  under  the  old  system  are 
still  in  development  and  remain  under  the  documents  produced 
by  that  system. 

Figure  3  depicts  the  documentation  and  review  procedure 
utilized  throughout  the  planning  phases  of  a  project. 
Proposals   of   future   concepts,   called    Advanced    System 
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FIGURE  3.  DOCUMENTATION  AND  REVIEW  PROCEDURE 
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Concepts  (ASC)  ,  are  contained  in  the  Navy  Advanced  Concepts 
(NAC)  .  Up  to  30  proposals  may  be  submitted  by  each  Systems 
Command  and  essentially  bridge  the  gap  between  science  and 
technology  and  a  budgeted  project.  The  Director  of  RDTSE 
within  the  Navy  selects  a  number  of  these  during  the  Navy 
budgeting  cycle  (POM  Process)  for  initiation  of  Advanced 
Development. 

Concise  statements  of  operational  needs  for  0-  10  years 
are  reflected  in  Operational  Requirements  (OR) .  These  three 
page  requests  for  proposal  from  CNM  are  originated  within 
OPNAV   at  the  request  of  Fleet  activities  or  other  commands. 

After  technical  studies,  CNM  replies  to  OPNAV  with 
alternative  solutions  contained  in  Development  Proposals 
(DPs).  OPNAVINST  5000.42  (Ref.  9)  suggests  that  an 
iterative  process  should  be  developed  between  the  OPNAV  OR 
sponsor  and  the  CNM  to  prepare  the  DP  and  resolve  all 
relevant  questions. 

In  response  to  the  DP/  OPNAV  prepares  a  Navy 
Development  Concept  Paper  (NDCP)  for  approval  by  the 
Acquisition  Review  Committee  (ARC) ,  a  sub-committee  of  the 
CNO  Executive  3oard  (CE3) .  The  NDCP  represents  the  first 
commitment  of  OPNAV  RDTSE  resources.  It  includes  issues, 
objectives,  plans,  parameters,  areas  of  risk,  and 
development  alternatives,  and  provides  the  basis  for  a 
Development  Concept  Paper  (DCP) .  The  DCP,  as  stated  in 
OPNAVINST  3930.83  (Ref.  10)  is  the  basic  source  of  advance 
evaluation  planning  information. 

Although  the  DCP  and  DSARC  process  is  utilized  only  for 
major  programs,  similar  processes  are  used  for  minor 
programs.  A  major  program  is  one  having  an  estimated  RDTSE 
cost  in  excess  of  $50  million,  an  estimated  production  cost 
greater   than   $200   million,   or   othenwise   designated   by 
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SECDEF.  Most  DCPs  cover  the  decision  the  Secretary  of 
Defense  is  being  asked  to  make,  the  alternatives,  the 
program  justification,  the  program  management  plan, 
projected  costs,  risks,  and  the  limits  of  the  grant  of 
authority  from  SECDEF  (thresholds) .  The  DCP  is  limited  to  20 
pages  and  is  essentially  a  contract  between  SECDEF  and  the 
Navy.  Programs  designated  by  the  Deputy  Director  of  Research 
&  Engineering  or  other  appropriate  principal  of  DSARC  are 
similarly  documented  by  Program  Memorandum  (PM) .  Each  is 
developed  through  interaction  of  DDR&E,  CNO,  CNM  and  the 
Principal  Developing  Activity  (PDA) ,  which  is  a  field 
installation. 

The  life  cycle  of  a  weapons  system,  as  defined  by 
DODDIR  5000.1  (Ref.  2)  consists  of  5  phases.  They  are: 
Conception,  Validation,  Full  Scale  Development,  Production, 
and  Deployment. 

The  Conception  Phase,  conducted  at  the  discretion  of 
the  Department  of  the  Navy,  establishes  a  technological, 
military,  and  economic  base  for  the  program  through  analysis 
and  experimentation.  It  culminates  with  the  approval  of  an 
Advanced  Systems  Concept. 

The  Validation  Phase,  often  identified  with  advanced 
development,  consists  of  hardware  development  and  extensive 
analysis  of  the  concepts  of  Phase  I.  Ideally,  it  ends  with 
a  successful  demonstration  of  a  "brass  board"  model. 

In  the  Full  Scale  Development  Phase  the  weapon  system 
and  all  support  items  are  designed,  fabricated,  and  tested. 
It  includes  TSE  both  on  the  part  of  the  contractor  and  the 
Navy. 

During  the  Production  Phase,  the  weapon  and  all  support 
itens  and  spares  are  produced  for  operational  use. 
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Th-e  final  phase,  Deployment,  consists  of  providing  the 
weapon  system  to  operational  units  and  its  subsequent  use  by 
them. 

In  view  of  the  above  life  cycle,  the  DCPs  are  aimed  at 
major  milestone  decisions  between  phases.  The  first 
decision,  DSARC  I,  comes  after  the  Conceptualization  Phase 
and  is  essentially  a  program  initiation  decision.  DSARC  IIA, 
initiation  of  Full  Scale  Development,  confirms  that  the  need 
for  the  system  is  current  and  that  development  risks  have 
been  identified  and  provided  with  solutions.  The  production 
decision  may  be  either  for  a  limited  (pilot)  production 
(DSARC  IIB)  or  full  scale  production  (DSARC  III) .  The  major 
elements  of  this  decision  are:  the  need  is  still  current,  a 
practical  engineering  design  has  been  achieved,  technical 
uncertainties  are  resolved,  TSE  has  determined  operational 
suitability,   and  the  remainder  of  the  program  is  realistic. 

DSARC  membership  includes  DDR&E,  the  Assistant 
Secretaries  of  Defense  for  Installations  and  Logistics  and 
Comptroller,  and  the  Director  of  Defense  Program  Analysis 
and  Evaluation  (DDPA&E) .  The  Deputy  Secretary  of  Defense  and 
other  ASDs  may  be  included  as  required. 

The  DCP  eventually  forms  the  basis  for  the  Project 
Manager's  overall  supervision  of  a  specific  project.  The 
Test  and  Evaluation  Easter  Plan  must  be  carefully 
constructed  to  ensure  that  all  key  issues  in  the  DCP  are 
addressed  and  all  questions  are  answered.  Thus  the  DCP 
becomes  an  important  document  in  determining  where  and  how 
the  TSE  for  the  project  will  be  conducted. 

NAVMATINST  3960.6  (Ref.  11)  states  that  the  TEMP  should 
be  considered  as  a  planning  document  rather  than  a  control 
document.  It  contains  objectives,  issues,  responsibilities, 
interfaces,   and   schedules.    It   attempts  to  identify  test 
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activities  and  facilities  intended  to  be  used  and  tasks  the 
PM  with  ensuring  all  necessary  liaison  and  coordination  with 
the  test  ranges  and  activities  is  accomplished  to  facilitate 
advance  planning.  The  instruction  requires  the  TEMP  to  be 
distributed  to  the  CNO  Program  Coordinator  (or  OPNAV 
sponsor),  OP-983,  COMOPTEVFOR,  CNM  (MAT  03L , 030, 033 , 04H,0UT)  , 
*  and  participating  Systems  Commands  and  field  activities.  The 
provisions   of  this  instruction  apply  to  all  major  projects. 

NAVAIRINST  3960.2  (Ref.  12)  places  the  requirement  for 
a  TEMP  on  Less-Than-Ma jor  (LTM)  Projects  (less  than  $50 
million  RDT&E,  less  than  $200  million  production  costs)  and 
Lower-Range-Less-Than-Ma jor  (LRLTM)  Projects  (less  than  S5 
million  RDT&E,  less  than  $20  million  production  costs)  .  A 
simplified  format  is  provided  for  LRLTM  Projects. 

Within  NAVAIR,  NAVAIRINST  5000. UA  (Ref.  13)  established 
an  integrated  NAVAIR  Naval  Aviation  Plan  (NNAP)  for  current, 
mid-range,  and  long-range  planning  within  estimated  fiscal 
constraints.  The  NNAP  is  designed  to  identify  worthwhile  new 
ideas  early  through  a  forecast  of  the  state-of-the-art  in 
critical  sciences  and  identification  of  operational 
requirements.  Specifically,  it  integrates  aircraft,  weapons 
and  carrier  planning  data  into  a  long  range  plan  for 
aircraft  and  weapon  development,  procurement,  and  support. 
Additionally,  it  identifies  critical  advanced  development, 
and  provides  the  Navy  with  long  range  logistic  and  facility 
requirements.  The  inherent  security  classification  of  the 
NNAF  renders  it  relatively  ineffectual  for  use  at  the 
facility. 

The  Project  Master  Plan  (PMP) ,  developed  by  the  Project 
Manager/Acquisition  Manager,  depicts  the  integration  of  all 
aspects  of  weapon  system  acquisition  such  as  ILS  (integrated 
logistic  support),  R&D,  Di'&E,  OT&E,  procurement  and  funding. 
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An  AIRTASK  is  utilized  within  NAVAI3  for  assigning  work 
to  field  activities.  NAVAIRINST  3900.8  (Ref.  14)  states  that 
an  AIRTASK  is  the  document  for  translating  the  budget  task 
into  work  assignments  to  be  performed  ny  a  field  activity. 
When  feasible,  it  includes  all  necessary  technical  details 
to  enable  field  activities  to  accomplish  the  work  in 
accordance  with  NAVAZR  requirements.  A  WORK  UNIT  ASSIGNMENT 
is  a  detailed  assignment  to  a  field  activity  for  performance 
of  a  specific  task  within  the  scope  of  a  previously  assigned 
AIRTASK. 

At  the  CSD  level,  in  addition  to  the  DCP/DSARC  process, 
other  types  of  R&D  coordinating  papers  exist.  They  are:  the 
Technology  Coordinating  Papers  (TCP)  which  •  highlights 
service  gaps,  overlaps  and  needs  in  a  given  technological 
area.  Area  Coordinating  Papers  (ACP)  which  are  similar  to 
TCPs  only  cover  warfare  specialty  areas,  and  Mission 
Coordinating  Papers  (MCP)  which  are  an  expansion  on  ACPs  and 
analyze  threat,  needed  technology,  force  implications, 
resource  impact,  and  future  options. 

C.    TEST  AND  EVALUATION 

The  major  objective  of  TSE  is  to  provide  the  best 
information  available  on  the  military  utility  of  a 
prospective  system,  its  operational  effectiveness  and 
operational  suitability,  needs  for  further  development  or 
modification,  and  data  useful  in  estimating  the  probaDle 
cost  of  development  completion,  acquisition  and  ownership. 

Basic  policy,  as  stated  in  Reference  1,  dictates 
evaluation  by  an  independent  organization  so  far  as  possible 
and  evaluation  of  operational  suitability  prior  to 
production.  Both  of  these  policies  are  primarily 
accomplished  through  the  conductance  of   I0T&E   by   OPTEVFOR 
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and   the   conductance   of   acceptance  trials  by  the  Board  of 
Inspection  and  Survey  (BIS) .   These  two  groups  report  to  the 
CNO    for    these   functions.    OPNAVINST   3960.8   (Ref.   5) 
recognizes  the  similarity  of  these  2  functions  and  calls  for 
close  coordination  to  prevent  unnecessary  duplication. 

The  mission  of  the  Board  of  Inspection  and  Survey  is, 
as  stated  in  OPNAVINST  5420.70: 

"...  to  inspect  new  model  aircraft  for 
construction  and  suitability  for  the  purpose 
intended  and  to  make  recommendations  on  their 
acceptance  by  the  Navy  ". 

The  Sub-Board  of  Inspection  and  Survey  at  Patdxent 
River  is  assigned  the  responsibility  of  conducting  trials  of 
aircraft  to  determine  if  the  aircraft  has  been  constructed 
in  accordance  with  contract  specifications.  All  materiel, 
performance  and  design  defects  and  deficiencies  found  are 
reported  by  the  Board  along  with  recommended  design  changes. 

T&E  is  integrated  throughout  the  acquisition  cycle  and 
is  not  merely  a  follow-up  to  R&D.  OSD  classifies  RDTSE  into 
two  categories:  Development  Test  and  Evaluation  and 
Operational  Test  and  Evaluation. 

DODDIR  5000.3  defines  DT&E  as: 

"...test  and  evaluation  conducted  to:demonstrate 
that  the  engineering  design  and  development 
process  is  complete;  demonstrate  that  the  design 
risks  have  been  minimized;  demonstrate  that  the 
system  will  meet  specifications;  and  estimate  the 
system's  military  utility  when  introduced." 

DT&E    includes    Engineering   Development   and   Design 
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Support  Tests,  Developing  Agency  (DA)  Tests  performed  by 
OPTEVFOR,  Contractor  Demonstration,  and  Technical  Evaluation 
(TECHEVAL).  Included  within  the  TECHEVAL  is  the  Navy 
Preliminary  Evaluation  (NPE) ,  an  early  assessment  of  mission 
potential  prior  to  extensive  tests,  and  the  Navy  Technical 
Evaluation  (NTE) ,  a  test  designed  to  determine  a  system1 s 
acceptability  for  fleet  use.  A  TECHEVAL,  as  stated  in 
OPNAVINST  3930. 8B,  is  assigned  by  CNO  in  response  to  a 
favorable  program  full-scale  development  decision  by  the 
SECDEF  or  comparable  CNO/CNH  decision  approving  the 
commitment  of  resources  for  full-scale  development. 
Additionally,  a  TECHEVAL  may  be  requested  by  a  Developing 
Activity  to  determine  suitability  for  other  acquisition 
programs  and  improvements  such  as  conversions  and 
modernizations. 

NAVHATIIIST  3960.6  (Ref.  11)  states  that  OPTEVFOR 
participation  in  DT&E  is  necessary  to  insure  that  meaningful 
and  realistic  tests  from  the  fleet  operational  viewpoint  are 
included  in  order  to  permit  an  early  assessment  of 
operational  suitability. 

OTSE  is  defined  as: 

"...that  test  and  evaluation  conducted  to  estimate 
the  prospective  system's  military  utility, 
operational  effectiveness,  and  operational 
suitability  (including  compatibility, 
interoperability,  reliability,  maintainability  and 
logistic  and  training  requirements) ,  and  need  for 
any  modifications.  In  addition,  OTSE  provides 
information  on  organization,  personnel 
requirements,  doctrine,  and  tactics.11 

An  important  subcategory  of  OT&E  is  Initial  Operational 
Test  and  Evaluation  (IOTSS)  which  according  to  DOD  directive 
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must  be  accomplished  prior  to  the  first  major  production 
decision.  IOTSE  is  conducted  through  an  Operational 
Evaluation  (OPEVAL) ,  which  is  assigned  by  CNO  at  the  tirce 
the  TECHEVAL  is  assigned.  The  objectives  of  OPEVAL  are  to 
detetmine  that  the  system  meets  program  objectives  under 
operational  conditions,  can  be  effectively  operated  and 
maintained  by  fleet  personnel,  logistic  support  is  feasible, 
and  all  performance  and  suitability  questions  are  adequately 
examined. 

OPNAVINST  3930. 8B  (Ref.  10)  defines  the  various 
priorities  which  will  be  assigned  to  projects  by  CNO.  They 
are:  AA-critical  to  the  Navy's  readiness  for  war,  A-projects 
of  an  urgent  nature  bearing  significantly  on  the  Navy's 
readiness  for  war,  and  B-projects  that  are  to  be  started  and 
completed  as  soon  as  higher  priority  operations  permit. 
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III.  £OINT  MUG  IPS  ROLE  IN  TE.ST  AND  EVALUATION 

This  chapter  follows  the  organizational  development  of 
a  TSE  activity,  namely,  the  Pacific  Missile  Test  Center, 
Point  Mugu.  Many  of  the  organizational  problems  experienced 
at  the  Point  Mugu  complex  are  similar  to  those  at  other  TSE 
activities,  since  they  are  all  operating  under  the  same 
environmental  influences.  A  brief  historical  background  of 
Point  Mugu's  past  activities  is  presented.  This  is  followed 
by  a  more  detailed  description  of  events  that  occurred 
during  the  past  few  years.  Some  of  the  problems  affecting 
Point  Mugu's  operational  effectiveness  are  discussed 
together  with  a  description  of  the  new  reorganization, 
recently  instituted  at  Point  Mugu. 

This  chapter  provides  an  appreciation  for  the 
difficulties  encountered  in  managing  the  operations  of  a  TSS 
installation  and,  hopefully,  will  provide  a  better 
perspective  of  the  T&E  evolution  from  the  activities' 
viewpoint.  it  must  be  emphasized  that  many  of  the 
criticisms  and  problems  mentioned  are  strictly  the  opinions 
of  the  authors.  These  opinions  were  derived  from  personal 
interviews  conducted  during  trips  made  to  the  Point  Mugu 
complex. 


A.  BACKGROUND 

The  Navy's  post  World  War  II  interest  in  Point  Mugu 
began  in  1945  when  a  jroup  of  Navy  engineers  and  technicians 
was  located  there  to  establish  a  Navy  weapons  development 
and  test  capability.  shortly  thereafter,  the  Naval  Missile 
Test  Center  (NAMTC)  was  established  to  carry  out  the  Navy's 
weapons  test  requirements.  in  1958,  with  growing  emphasis 
on  space  programs,  the  range  organizations  of  NAMTC  were 
reorganized   into   the   Pacific  Missile  Range  and  designated 


32 


one  of  the  six  National  Ranges  under  the  administration  of 
Deputy  Director,  Test  and  Evaluation,  DoD,  with  the  Navy 
designated  as  its  flange  Management  Agency  (RMA) ,  responsible 
for  management  and  operation.  Under  the  National  Range 
concept,  all  DoD  and  other  federal  agencies  obtained  common 
ranga  support  services  free,  with  the  exceptions  of  services 
performed  outside  the  normal  working  day  or  services  that 
reguired  extraordinary  data  reproduction  (Ref.  15)  .  The 
remainder  of  the  NAMTC  organization  was  consolidated  into 
the  Naval  Missile  Center,  as  a  separate  field  activity  of 
the  then  Bureau  of  Aeronautics.  Both  NMC  and  PMR  were 
separate  activities  hosted  aboard  the  Naval  Air  Station,  pt. 
Mugu,  which  was  a  subordinate  command  of  the  PMR.  NMC 
maintained  its  role  in  weapons  test  functions  utilizing  the 
range  services  of  the  PMR  which  played  an  important  role  in 
satellite  and  space  vehicle  programs  of  the  SO's. 

As  funding  for  the  space  program  peaked  out  and  with 
growing  public  concern  over  the  the  Vietnam  conflict, 
emphasis  at  the  PMR  again  shifted  to  weapons.  NMC  became  the 
primary  user  of  range  services  at  PMR  as  test  and  evaluation 
took  a  significant  role  in  weapons  development.  in  1970, 
NMC  was  designated  as  a  subordinate  command  under  the 
Commander  of  PMR  (Ref.  16).  PMR's  command  authority  was  only 
perfuncory  as  NMC  struggled  to  maintain  its  identity  as  a 
separate  coamand.  In  June  1974,  NMC  was  again  established 
as  an  echelon  4  activity  (subordinate  to  Systems  Commands) , 
reporting  directly  to  NAVAIRSYSCOM  and  hence  bypassing 
COMPMR  in  the  chain  of  command.  NMC's  separate  command 
identity  was  shortlived,  however,  as  COMPMR  successfully 
convinced  NAVAIRSYSCOM  that  the  Pt.  Mugu  complex  would  be 
more  effective  if  the  resources  of  NMC  and  PMR  were 
reorganized  into  a  separate  command  activity,  the  Pacific 
Missilie  Test  Center  (PMIC) . 
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B.  NMC'S  INVOLVEMENT 

In  1958,  with  the  establishment  of  PMR  as  a  National 
Range,  NMC  began  to  expand  its  capabilities  as  a  weapons 
test  facility  which  entailed  functions  already  available  at 
PHfi.  By  1970,  NMC  was  capable  of  providing  total  weapons 
testing  services  with  the  exception  of  the  range  resources 
needed  for  actual  launches  {i.e.  launch  pads  and  range 
terrain) .  The  services  at  NMC  included  the  planning  and 
conducting  of  engineering  experiments  for  determining  the 
operational  effectiveness  and  reliability  of  assigned 
weapons  systems  as  well  as  the  laboratory  investigations 
concerning  the  development,  test,  and  evaluation  of  weapon 
systems.  Other  functions,  however,  not  normally  performed  by 
a  weapons  testing  facility  had  also  been  developed.  The 
Aircraft  Maintenance  Department  accomplished  intermediate 
level  maintenance  on  the  growing  fleet  of  aircraft  assigned 
to  NMC.  The  Photo/Graphics  Department  .  had  developed 
photographic  facilities  which  constituted  one  of  the 
best-equipped  laboratories  in  the  Navy.  The  Threat 
Simulation  Department  had  developed  expertise  in  the  design, 
development,  procurement  and  operation  of  aerial,  seaborne, 
land  and  special  targets. 

C.  CONSOLIDATION 

By  1970  PMR's  space  program  involvement  was  all  but 
eliminated  by  the  increasing  emphasis  on  weapons  testing. 
For  purposes  of  reducing  duplicative  functions  of  PMR  and 
NMC  both  activities  were  placed  under  one  commander  along 
with  the  Naval  Air  Station  and  the  PMR  Facility,  Hawaiian 
Area  (Figure  4).  NMC  would  provide  photographic  and 
aircraft  maintenance  services  for  tenant  activities  and 
provide  target  development  and  operation  support  for  the  PMR 
Directorate. 
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PMR,  as  a  prime  supporter  of  the  space  program  had 
found  little  difficulty  obtaining  funding  for  range 
expansion  and  improvement.  PMR's  institutional  funding 
flowed  directly  from  the  DoD  level  and  resource  allocation 
was  loosely  controlled.  As  a  member  of  the  National  Ranges, 
PMR's  facilities  were  available  "f ree-of-charge"  and, 
consequently,  little  motivation  was  provided  to  improve 
efficiency.  Many  observers  at  the  Pt.  Mugu  complex  have 
described  PMR  as  a  stagnant  organization.  One  individual 
who  was  attached  to  the  range  organization  of  NAMTC  back  in 
the  50's  and  had  recently  rejoined  the  organization  stated  " 
The  range  is  still  conducting  business  the  way  it  did  20 
years  ago."  NMC,  on  the  other  hand,  was  receiving  much  of 
its  funding  through  Navy  weapon  systems  appropriation 
categories.  Tighter  control  of  resources  through  the  Navy 
chaia-of-command  had  a  positive  effect  on  NMC's  efficiency 
since  NMC  enjoyed  a  favorable  reputation  as  a  well-run, 
dynamic  organization.  Command  assignment  to  NMC  was  highly 
desirable  and  almost  an  assurance  for  promotion.  Assignment 
to  command  of  PKR  was  not  viewed  in  the  same  light  as, 
traditionally,  the  commander  of  PMR  was  on  his  "twilight 
tour41. 

With  the  exception  of  eliminating  a  number  of 
duplicative  functions  the  consolidation  did  little  more  than 
to  bring  the  Commanding  Officer  of  NMC  under  the 
chain-of-command  of  the  Admiral  stationed  at  the  Point  Mugu 
complex. 

1 •  Bnyirpngental  Influences 

Almost  concurrently  with  the  1970  Point  Mugu 
consolidation,  NMC  was  placed  under  the  Navy  Institutional 
Funding  (NIF)  system.  NMC,  as  a  NIF  activity,  would  require 
customers  to  pay  for  the  total  cost  of  services  provided, 
direct   cost   plus   a   share   of  the  overhead.  PMR,  however, 
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woull  remain  solely  institutionally  funded  and 
"f ree-of-charge"  services  would  continue  for  range  users. 
NMC  recognized  the  new  funding  technigue  as  a  means  of 
expanding  and  improving  existing  resources.  NMC  aggressively 
sougat  increased  business  and  actively  competed  with  other 
T&E  facilities  for  new  weapons  program  selections.  During 
the  period  of  197Q  to  1974  NMC  increased  its  annual  business 
activity  from  approximately  $40  million  to  over  $100 
million.  PUR,  on  the  other  hand,  had  little  control  over 
improving  its  existing  assets.  With  defense  appropriations 
shrinking,  PMR's  institutional  funding  leveled  off  at  $6  1 
million  during  the  same  period,  as  reported  in  its  FY  74 
budget  forecast.  Stagnation  continued  to  be  a  way  of 
business  at  PMR,  with  no  motivation  for  improved  efficiency. 

Since  the  consolidation,  unrest  and  turmoil  have 
hampared  operations  aboard  the  Pt.  Mugu  complex.  Lack,  of 
directipn  from  higher  authority  was  recognized  as  a  primary 
source  of  discontent.  Coordination  of  TSE  activities  was 
nonexistent.  As  a  result,  7SE  field  activities  functioned 
autonomously,  lacking  clear  objectives  on  where  to 
concentrate  their  efforts.  Fearing  exclusion  from  new 
programs,  TSE  activities  attempted  to  keep  their  technology 
base  broad  enough  to  permit  inclusion  in  future  programs. 
While  NMC,  under  its  NIP  funding,  was  able  to  take  advantage 
of  the  competitive  environment  and  market  its  services  to 
potential  users,  the  lack  of  TSE  coordination  was 
detrimental  to  PMR.  PMR's  inclusion  in  a  program  had  to  be 
determined  early  enough  in  a  weapon  systems'  development  to 
provide  sufficient  lead  time  for  development  and 
installation  of  the  sophisticated  equipment  needed  for  the 
range.  Other  National  Ranges  possessed  enough  functional 
characteristics  similiar  to  PMR  to  threaten  its  exclusion. 
Since  funding  for  National  Ranges  was  through  appropriations 
categories  originating  at  the  DoD  level,  PMR  personnel  had 
little   influence   ovei   their   own    future.     Frustration 
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permeated  throughout  the  PMK  establishment.  Personnel  felt 
unable  to  measure  their  effectiveness  in  comparison  with 
other  range  activities  and,  consequently,  helpless  in 
determining  future  programs'  range  selections. 

Alth6ugh  the  majority  of  projects,  by  dollar  value, 
contracted  by  NMC  required  the  use  of  PMR  facilities,  there 
existed  a  definite  lack  of  cooperation  between  the  two 
activities.  Communications  flow  between  the  two  activities 
was  poor.  Each  command  maintained  its  own  contact  point  for 
external  inquiries  on  the  progress  of  the  same  program. 
Duplicative  functions  again  grew.  In  one  instance,  both 
commands  were  developing  a  data  instrumentation  package  for 
the  same  purpose  without  each  other  being  aware  of  it.  By 
disassociating  itself  from  PMR,  NMC  was  able  to  disclaim 
responsibility  for  inadequacies  arising  during  the  range 
phases  of  the  T&E.  The  reason  for  the  rivalous 
relationship,  rather  than  a  cooperative  atmosphere,  wasn't 
clear  but  perhaps  NEC's  insistence  on  maintaining  its 
separate  command  identity  was  to  blame. 

D.  REORGANIZATION  PROPOSED 

In  1972,  the  new  Commanding  Officer  of  PMR  recognized  a 
need  to  set  aside  the  growing  unrest  that  was  affecting  the 
entire  organization.  He  cited  that  lack  of  direction  from 
above  was  hindering  PMR's  ability  to  evaluate  itself.  No 
clear  dividing  line  of  responsibility  between  Washington  and 
field  activities  existed.  A  series  of  ad  hoc  committees  were 
formed  to  investigate  the  difficulties  existing  at  the  Point 
Mugu  complex,  while  the  Command  focused  its  thrust  upon 
stirring  up  action  in  Washington.  Shortly  thereafter,  the 
new  Commanding  Officer  announced  that  he  was  proposing  a 
reorganization  of  Pt.  Mugu.  External  support  for  the 
reorganization  was  gained  by  claiming  that  over  300  jobs 
could   be   eliminated   through  the  reorganization.   Internal 
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pressure  against  the  proposal  mounted.  Personnel  production 
suffered  due  to  the  uncertainty  of  job  security.  Tne 
command  billet  at  NMC,  which  was  a  popular  career  enhancing 
billet  normally  held  by  an  Aeronautical  Engineering  Duty 
Officer  (AEDO)  ,  was  also  threatened  by  possible  extinction. 
Elimination  of  the  billet  was  strongly  opposed  by  the  AEDO 
community. 

Before  the  reorganization  obtained  enough  support  for 
implementation,  another  shock  wave  infiltrated  the  complex. 
Early  in  1973,  the  Shore  Establishment  Realignment  (3ER) , 
which  was  successful  in  closing  the  Hunters  Point  Shipyard, 
at  the  cost  of  over  4000  jobs,  announced  plans  to  turn  the 
range  functions  at  PMR  over  to  private  contractors,  thus 
eliminating  T400  jobs.  The  employee  organization  responded 
vigorously  to  squelch  this  latest  proposal  which  was 
eventually  defeated  by  Congress.  Meanwhile,  the  SER 
incident  had  dominated  the  scene  through  1973  and,  the 
reorganization  never  became  a  reality.  Due  to  the  opposition 
from  both  within  and  external  to  the  organization  there 
existed  strong  doubts  that,  even  without  the  SER,  the 
reorganization  would  have  been  accomplished. 

E.  LATEST  EVENTS 

By  1974,  it  became  increasingly  obvious  to  the 
Washington  based  Navy  hierachy  that  the  consolidation  of  PMR 
in  1970  had  accomplished  very  little  towards  increasing  the 
complex's  effectiveness.  With  the  exception  of  eliminating 
some  duplicative  functions,  the  two  primary  activities  at 
Point  Mugu  had  suffered  considerable  internal  strife.  The 
arrival  of  a  new  Commanding  Officer  at  Point  Mugu,  in  June 
of  1974,  was  overshadowed  by  two  significant  events:  1)  NMC 
was  no  longer  in  the  chain- of-command  of:  the  COMPMR,  but 
instead,   was  reinstated  as  an  echelon  4  activity,  reporting 
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directly  to  NAVAIRSY5C0M}  2)  The  National  Ranges  would  now 
charge  users  all  direct  costs  associated  with  their 
projects. 

The  reestablishment  of  NMC  as  an  echelon  4  activity 
reasserted  the  apparent,  that  NMC  was  indeed  a  separate 
field  activity.  The  fact  that  NHC's  funding  authorization 
flowed  directly  from  NAVAIRSYSCOM  had  seriously  hindered 
COMPMR's  ability  to  exercise  any  real  control  over  NMC's 
operations.  Whether  or  not  NMC's  "new  found"  autonomy  would 
have  a  positive  effect  on  Point  Mugu's  effectiveness  was 
questionable. 

The  direct  cost  to  customers  scheme  would  certainly 
have  an  effect  on  PHR's  mode  of  operation.  Not  only  would 
range  users  be  able  to  assess  ranges  on  the  less  tangible 
characteristic  of  service  quality  but,  also,  by  the  more 
objective  method  of  service  cost.  Whether  or  not  PMR  could 
successfully  respond  to  the  new  challenge  of  keeping  costs 
down  was  uncertain,  but  what  was  certain  is  that  PMR  was 
given  a  motivation  for  efficiency  and  an  opportunity  to 
evaluate  itself  relative  to  other  ranges.  The  stagnancy 
that  degraded  PMR's  effectiveness  in  the  past  wasn't 
tolerable  under  the  new  funding  method,  which  would  instill 
a  competitive  operating  mode  among  the  various  ranges  for 
the  TSE  dollar.  Not  only  would  cost  efficient  operations 
mean  more  ousiness  activity  for  PMR,  but  also  would  weigh 
significantly  in  the  amount  of  institutional  funding  flowing 
from  DoD  for  expansion  and  improvement  of  facilities. 
General  Starbird,  DD(TkE),  DoD,  has  closely  monitored  the 
allocation  of  institutional  funding  for  the  various  T&E 
activities  and  has  been  a  strong  advocate  for  efficiency 
measurements  of  TSE  facilities.  He  has  made  his  opinion 
quite  clear;  the  cost  efficient  facilities  will  receive  the 
dollars.  PMR  personnel  have  recognized  the  impact  of  the 
funding    change   and   have   initiated   efforts   to   include 
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marketing  in  their  mode  of  operation. 

No  sooner  had  the  events  of  June  1974  been  absorbed 
when  the  new  COMPMR  released  startling  news.  He  felt  the 
present  organization  wasn't  responsive  to  the  Navy's  T&E 
needs  and  that,  consequently,  a  reorganization  would  be 
studied.  "the  changes  proposed  by  the  Admiral  were  radical 
and  would  totally  eliminate  NMC's  identity.  To  appease 
opposition  from  the  AEDO  community,  he  proposed  that  an  AEDO 
would  hold  the  second  highest  billet,  vice  commander,  and  he 
would  have  responsibilities  commensurate  with  the  NMC 
command  position.  Internal  resistance  never  surfaced,  as 
with  the  previous  reorganization  proposal,  since  no  jobs 
were  proposed  for  elimination.  Washington's  support  was 
achieved  by  selling  the  benefits  of  a  reorganized  Point  Mugu 
complex  and  avoiding  any  inferences  to  inadequacies  existing 
at  upper  echelon  operating  levels.  The  new  organization  was 
to  be  named  the  Naval  Weapons  Test  Center  (NWTC)  thus 
avoiding  any  inference  to  the  PMR.  The  reorganization  gained 
acceptance.  However,  just  prior  to  finalization  General 
Starbird,  a  strong  opponent  of  service  parochialism, 
objected  to  the  use  of  Naval  in  the  title.  Navy  officials 
acquiesced,  and  on  April  25,  1975,  the  Pacific  Missile  Test 
Center  (PMTC)  was  established. 

The  1970-1974  timeframe  at  Point  Mugu  was  not  a 
productive  period  in  terms  of  efficiency.  Internal 
managerial  problems  and  lack  of  external  guidance  are 
identified  as  primary  reasons.  The  PMTC  organization  is  an 
attempt  to  solve  many  of  the  internal  problems  and  is 
structured  to  be  more  responsive  to  external  activities. 


F.  PACIFIC  MISSILE  TEST  CENTER 


Specific    documents    addressing     the    need     for 
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reorganization  of  the  Point  Mugu  complex  were  not  available 
for  general  dissemination.  However,  several  studies  in 
recent  ,  years  have  identified  the  need  for  improved 
organizational  effectiveness  of  T&E  facilities  in  general 
(Refs.  17,  18,  and  19).  The  data  justifying  a 
reorganization  is  limited  to  personal  interviews  of 
personnel  at  Point  Mugu  and  the  subjective  conclusions 
stemming  from  these  interviews. 

1.  Weed  For  Reorganization 

The  following  is  a  list  of  specific  problems 
recently  identified  as  affecting  the  overall  effectiveness 
of  the  Point  Mugu  complex  which,  unless  otherwise  noted, 
refer  to  the  range  organization. 

a.  Lack  of  Technical  Advice 

The  mode  of  operations  on  the  range  has  been 
technically  passive.  The  user  requested  a  certain  service 
and  received  whatever  the  range  was  capable  of  providing. 
The  technical  competence  exists  at  the  range  to  advise  the 
user  of  what  he  needs  rather  than  what  he  wants.  Range 
personnel,  familiar  with  its  limitations,  could  most 
adequately  accomodate  a  user's  requirements  by  tailoring 
need  to  match  the  range's  capabilities. 

b.  Lack  of  Cooperation 

Both  the  range  and  the  missile  center  have  the 
same  basic  mission  of  supporting  TS2.  As  such,  both  must 
operate  on  a  cooperative  basis  for  without  one  there  would 
hardly  be  a  need  for  the  other.  However,  as  the  previous 
development  has  shown,  this  was  not  the  case. 

c.  Unclear  External  Perception 

To  the  user  who  was  having  T&E  performed  at 
Point   Mugu   there   was  little  distinction  between  the  ranje 
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and  the  missile  center.  When  a  question  arose  that  needed 
to  be  answered  immediately,  he  frustratedly  discovered  that 
he  would  have  to  make  a  distinction  between  which  activity 
to  direct  his  inquiry  since  both  activities  maintained 
separate  contact  points  on  projects  and  made  little  attempt 
to  communicate  with  each  other.- 

d.  Staff  Duplication 

This  challenges  the  need  for  duplicative  staffs 
at  two  activities,  located  at  the  same  geographical  point, 
whose  functions  are  complementary. 

e.  Long  Range  Planning  Conflicts 

It  is  hard  to  believe  that  activities  at  this 
level  can  justifiably  plan  for  their  future  involvement  in 
particular  weapon  systems  when  that  deterrainat ioion  is 
dependent  entirely  on  decisions  made  at  higher  echelons. 
However,  lack  of  guidance  from  higher  authority  has  forced 
PMR  &  NMC  to  plan  their  future  involvements.  Unfortunately, 
both  activities  do  this  independently,  failing  to  recognize 
their  own  vested  interest. 

f.  Personnel  Stagnation 

A  common  complaint  in  any  organization  with 
tenure  status  is  the  problem  of  how  to  deal  with  tenured 
personnel  who  cease  to  contribute  useful  benefit  to  the 
organization.  Key  managerial  positions  have  been  filled  by 
the  same  people  for  many  years  who  have  resisted  changes  to 
their  mode  of  operation.  One  person  stated  "When  a  RTF 
(Reduction  in  Force)  occurs,  the  best  people  must  go  and 
we're  left  with  the  deadwood.  The  only  thing  wa  can  do  is 
try  and  build  the  organization  around  them." 

g.  Empire  Building 

Personnel  stagnation  has  also  fostered  "empire 
building"    with   separate   units   within   the   organization 


4  3 


functioning  almost  independent  of  the  Command.  One 
department  promulgated  its  own  five  year  plan  while  the 
s-taff  prepared  one  for  the  department  also. 

h.   Improper  procedural  Implementation 

Changes  were  generated  at  the  Command  level  and 
presented  to  the  executive  board  for  dissemination. 
Functional  managers  did  not  participate  in  policy 
implementation.  Ideas  flowed  strictly  down  the 
chain-of-command  without  a  feedback  mechanism. 

i.   Inadeguate  Reports 

The  management  information  system  generated  data 
inadeguate  for  managers  to  make  decisions.  Reports  were 
structured  in  financial  terms  only,  suitable  for  a  financial 
analyst's  interpretation.  As  a  consequence,  the  comptroller 
was  involved  in  making  decisions  which  should  be  made  by 
functional  managers.  The  comptroller  should  have  only  been 
responsible  for  the  operation  of  set  policy  and  not  engaged 
in  planning  decisions. 

j.   Inappropriate  Involvement 

NMC,  through  its  agressiveness,  has  inherited 
some  responsibilities  not  involved  with  their  primary 
mission.  At  times,  prooiems  in  these  areas  have  tended  to 
bog  down  the  Command's  effectiveness.  The  Threat  Simulation 
Department,  which  was  responsible  for  target  development 
belonged  under  the  cognizance  of  the  range  activity. 
Aircraft  maintenance  is  normally  the  responsibility  of  a 
Naval  Air  Station  and  not  of  a  T&E  activity. 

2-   Corporate  Characteristics 

The  PMTC  organization,  recently  established  at  Point 
Mugu,  was  a  radical  change  from  the  previous  organization. 
Many  of  the   new   innovations   incorporated   were   developed 
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following  a  comprehensive  presentation  from  the  Vice 
President  of  Management  Systems  at  the  TRW  Corporation.  The 
dynamic  characteristics  of  TRW,  which  were  considered 
necessary  to  remain  competitive  in  the  rapidly  changing 
technological  environment,  were  also  considered  suitable  for 
PMTC's  interface  with  the  T&E  environment.  By  instituting  a 
TRW-like  organization,  PMTC  would  be  acquiring  both  the 
strengths  and  weaknesses  of  TRW's  organization. 

The  matrix  project  mangement  method  of  TRW  (Ref. 
20)  assigned  project  managers  from  the  functional 
departments  of  the  organization  to  manage  the  company's 
total  life  involvement  in  a  project.  Upon  completion,  the 
project  manager  returned  to  a  functional  assignment.  This 
had  proved  to  be  beneficial,  in  that  project  managers  were 
kept  abreast  of  the  latest  technology  and  personnel 
stagnation  was  avoided  in  project  management  positions.  On 
the  other  hand,  selection  to  a  project  management  position 
was  temporary  and,  once  the  project  was  completed,  there  was 
no  assurance  that  his  old  position  would  still  be  available. 
Consequently,  qualified  personnel  would  sometimes  shun  the 
opportunity  for  visibility  as  a  project  manager. 

The  matrix  project  mangement  method  also  provided 
for  efficient  allocation  of  personnel.  More  flexibilty 
existed  to  shuttle  personnel  to  work  on  other  projects  when 
not  needed  on  a  particular  one  and,  consequently,  eliminated 
a  large  amount  of  idle  manpower.  This  benefit  was  traded 
off  with  the  increased  cost  coliecrion  problem  of 
identifying  the  costs  associated  with  a  particular  project. 
Personnel  could  be  working  on  more  than  one  project  during 
the  the  same  period  and  the  problem  of  cost  identification 
increased.  One  of  the  reasons  for  successful  project 
completions  has  been  the  personal  integrity  of  the  personnel 
which  motivated  them  to  work  extra  hours  to  get  the  job 
done.  To  what  degree  personnel  at  PMTC  will  be  motivated 
has  yet  to  be  determined. 
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3«   The  Organization 

The  resources  of  the  Naval  Missile  Center  and  the 
Pacific  Missile  Range  were  combined  into  the  Pacific  Missile 
Test  Center  whose  mission  is  (Hef.  21): 

"To  perform  development  test  and  evaluation, 
development  support,  and  follow-on  engineering, 
logistic,  and  training  support  for  Naval  weapons, 
weapons  systems,  and  related  devices;  and  to 
provide  Major  Range,  technical,  and  base  support 
for  Fleet  users  and  other  Department  of  Defense 
and  government  agencies.11 

PMTC  is  comprised  of  seven  directorates  and  two 
management  groups  (Figure  5) .  Of  the  seven  directorates, 
three  are  directly  involved  with  the  functional  aspects  of 
current  T&E  projects  and  another,  the  Fleet  Weapons 
Engineering  Directorate,  follows  the  life  cycle  of  a  weapon 
system  after  it  has  completed  development  at  PMTC  and  is 
introduced  to  the  Fleet.  Two  other  directorates,  PMRF 
Hawaii,  and  the  Marine  Air  Detachment,  have  functions  not 
directly  affected  by  the  reorganization.  A  brief  discussion 
of  the  other  directorates  and  the  two  groups  follows. 

a.   Naval  Air  Station 

An  institutionally  funded  directorate  whose  main 
function  of  operating  and  maintaining  base  facilities  and 
provide  base  support  services  for  the  PMTC  and  ^assigned 
tenant  activities  will  remain  relatively  intact.  In 
addition,  NAS  assumed  responsibilit j  for  aircraft 
maintenance,  formerly  performed  by  NMC  The  Surface  Craft 
Department,  whose  function  was  to  operate,  maintain,  and 
provide  surface  craft  in  support  of  range  operations,  range 
surveillance,  and  area  clearance,  was  shixted  to  the  Range 
Directorate. 
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b.  Range  Directorate 

An  institutionally  funded  directorate  with  a 
strictly  operationally  oriented  function  that  provides 
assigned  Major  Range  services,  related  range  facilities,  and 
target  systems  for  PMTC  projects,  including  development  of 
range  systems,  subsystems,  and  technigues.  The  heart  of  the 
PMR,  the  Range  Directorate  charges  users  direct  costs  for 
services  rendered.  Released  from  all  non-operational 
functions,  the  directorate  assumed  responsibility  for  the 
Surface  Craft  Department's  functions  of  NAS  and  the  Threat 
Simulation  Department's  functions  of  NMC. 

c.  Fleet  Support  Directorate 

Formerly  the  Fleet  Weapons  Engineering 
Department  of  NMC,  this  directorate's  function  remained 
intact.  The  directorate  provides  the  single  entry  point  for 
all  PMTC  in-service  engineering  projects  after  completion  of 
test  and  evaluation  phases. 

d.  Systems  Evaluation  Directorate 

A  NIF  funded  directorate,  the  directorate 
provides  the  technical  expertise  needed  to  appraise  a 
customer's  T6E  requirements.  Comprising  many  of  the 
functions  of  NMC's  Laboratory  Department,  the  directorate 
performs  development  test  and  evaluation,  development 
suppDrt,  test  and  evaluation  support  to  in-service 
engineering  cf  Naval  weapons,  weapons  systems  and  related 
devices,  including  development  of  test  and  evaluation 
technologies  and  methodologies. 

e.  Engineering  Applications  Directorate 

A   NIF   funded   directorate,    the  directorate 

assumed    responsibility   for   the   functions  of   the   Data 

Analysis  and  Processing   Department   of   PMR,  some   of   the 

functions   of  the  Data  Analysis  and  Processing  Department  of 
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PMR,  and  some  of  NMC's  Laboratory  Departments  functions. 
Throagh  its  department,  the  directorate  supports  the 
protDtype  design  and  fabrication  needed  for  PMTC  projects  as 
well  as  photographic  and  instrumentation  requirements. 

f.  Project  Management  Group 

Through  this  group,  the  project  management 
matrix  technique  is  enacted.  The  group  represents  the 
single  entry  point  for  PMTC  projects  providing  project 
management,  coordination,  and  information  for  PMTC  projects.. 
For  adequate  customer  responsiveness,  the  group  has  mirrored 
the  structure  of  the  NAVAIRSYSCOM  Project  Managers1  offices, 
a  major  source  of  PMTC  projects.  Similar  to  the  autonomy 
enjoyed  by  NAVAIR  Project  Managers,  Project  Officers  work 
directly  for  the  Commander  PMTC,  while  obtaining 
administrative  support  from  the  PMG.  By  operating  in  the 
classical  matrix  mode,  the  Project  Officer  crosses 
horizontally  through  functional  lines  in  the  organization  to 
ensure  proper  project  development.  The  Project  Officer  is 
not  supposed  to  infiltrate  vertically  through  the 
organization  by  involving  himself  in  the  day-to-day  tasks 
which  are  the  responsibility  of  functional  task  managers. 
His  primary  concerns  are  time  and  cost.  Project  priority  is 
set  by  the  Director  of  the  PMG  and  thus  avoids  conflicts 
between  Project  Officers  and  functional  managers  over 
priorities,  traditionally  a  source  of  conflict. 

g.  Resources  Management  Group 

This  group  provides  a  unigue  aspect  to  the  PMTC 
organization  in  that  it  provides  a  new  approach  to  resources 
management.  The  group's  function  is  to  provide  policy 
guidance,  advice,  and  integrated  planning  and  information 
for  the  management  of  the  PMTC  financial,  workforce,  and 
physical  resources,  and  provide  support  to  all  PMTC 
management  levels.  By  it^.  method  of  operation,  the  group 
encourages     participative    involvement    throughout    the 
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organization  by  acting  as  a  focal  point  for  both  policy 
planning/implementation  and  resource  allocation.  The  RMG 
could  best  be  described  as  a  matrix  organization  in  reverse, 
distinguishing  itself  from  the  normal  matrix/staff 
organization.  An  example  would  best  serve  to  point  out  the 
differentiation: 

Under  a  matrix/staff  organization  a  department 
head,  citing  a  need  for  a  financial  analyst  in  his 
department,  would  set  about  to  hire  a  financial 
analyst.  After  finding  someone  that  appears 
suitable  for  the  position,  using  his  limited 
knowledge  of  finances  as  a  criterion,  he  hires  the 
person.  The  newly  hired  person  works  for  the 
department  and  is  evaluated  based  on  the  department 
head's  appraisal  of  his  work.  The  comptroller,  in  a 
staff  position,  has  no  direct  control  over  the 
analyst's  performance  and  exercises  minor  control 
through  the  department  head. 

Under  PMTC's  organization,  the  head  of  the  P.KG's 
Comptroller/Financial  Mangement  Department  cites  a 
need  for  a  financial  analyst  in  a  department.  All 
the  financial  analysts  work  directly  for  him  and, 
consequently,  he  is  able  to  sr.ift  them  wherever  he 
deems  appropriate.  Quite  often,  another  department 
may  have  an  overallowance  and,-  an  analyst  can  be 
shifted  to  the  department  needing  him,  thus 
precluding  the  hiring  of  an  additional  analyst.  The 
analyst  works  directly  for  the  comptroller, 
providing  financial  services  to  the  department. 

The  centralization  of  resources  has  great  advantages 
in  resource  allocation.  If  a  directorate  cites  a  need  for 
additional  office  space,  the  property  Management  Division  of 
the  RMG  can  objectively  evaluate  the  request  and  determine 
the  most  efficient  allocation  of  additional  office  space, 
considering  the  overall  benefit  to  PMTC.   Through   the   Work 
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Force  Management  Division,  personnel  talent  can  be  most 
effectively  distributed  throughout  the  organization.  It 
provides  a  vehicle  for  moving  people  between  directorates  to 
develop  competent  Project  Officers.  The  dynamics  of 
personnel  movement,  used  by  the  division,  can  also  be  used 
to  eliminate  the  problem  of  personnel  stagnation. 

The  centralization  of  resource  allocation  may  have 
taken  some  control  away  from  the  directorate  heads  but  the 
advantages  of  centralization  appear  to  outweigh  this 
drawback.  Looking  froni  another  viewpoint,  the  directorate 
heads  no  longer  need  concern  themselves  over  proper  resource 
allocation  and  can  direct  their  energies  to  the  prime 
objectives  of  their  directorate.  Within  each  directorate 
there  is  a  resources  office  which  is  responsible  for  the 
financial  functions  and  which  also  has  personnel  reporting 
to  the  RMG  on  matters  of  planning,  property  mangement, 
management  systems,  and  work  force  management.  The  planning 
aspect  of  the  RMG  provides  an  excellent  vehicle  for  two-way 
communications  and  policy/change  implementation.  The 
directorates,  through  the  RMG,  state  the  need  for  a  policy 
change  to  the  Executive  Board,  comprising  the  heads  of  all 
the  directorates  and  the  COMPMTC.  The  Board  reviews  'the 
proposals  recommended  by  the  RUG  and  makes  the  decision. 
The  RMG  then  disseminates  the  policy  guidance  and  manages 
its  implementation.  Through  the  RMG,  the  Command  hopes  to 
attain  participative  management  throughout  all  levels  of  the 
organization. 

G.   FINDINGS 

Many  of  the  problems  afflicting  Point  Mugu  have  been 
eliminated  by  virtue  of  the  reorganization  which  eliminated 
separate  command  identities.  Still  present  are  the  people 
who  were  involved  in  and  caused  many  or  the  problems  thit 
plagued  the  complex  for  so  many  years.   The  new  organization 


provides  a  better  means  for  the  Command  to  react  to  the 
changing  external  environment.  Internally,  the  organization 
possesses  the  dynamic  capability  to  adequately  assign  its 
resources  and  obtain  their  most  beneficial  contribution,. 
What  remains  to  be  seen  is  if  the  personnel  will  provide  the 
cooperation  and  the  flexibility  the  nev  organization 
demands.  Only  through  personnel  motivation  can  the  problems 
of  technical  advise  and  lack  of  cooperation  be  eradicated. 
The  external  perception  of  PMTC  can  best  be  improved  by 
customer  satisfaction,  again,  a  function  of  personnel.  The 
problem  of  inadequate  reports  is  still  being  addressed.  A 
centralized,  highly  elaborate  Management  Information  System 
(MIS)  is  proposed  that  will  provide  the  proper  information 
needed  at  each  level  of  management  to  put  the  decisions  back 
in  the  hands  of  the  functional  managers.  Inappropriate 
involvement  has  been  reduced  by  attempting  to  identify  the 
most  suitable  directorate  for  functions  and  by  eliminating 
those  functions  which  serve  no  useful  purpose  in  PMTC's 
mission.  The  EMG  provides  an  adequate,  policy  implementation 
and  feedback  mechanism,  enhancing  the  principles  of 
participative  management.  Additionally,  the  RMG  has  a 
rather  effective  method  of  eliminating  personnel  stagnation 
by  moving  personnel  out  of  key  positions  under  the  guise  of 
personal  development. 

Other  T&E  activities  are  also  recognizing  their 
organizational  inadequacies  and  are  presently  revampin ; 
their  organizations  for  more  efficient  operations.  NATC  has 
also  just  undergone  a  similar  type  of  reorganization.  Yet 
to  be  resolved  are  the  inadequacies  of  upper  echelon 
direction  and  coordination.  A  separate  command  at 
NAVAIRSYSCOM  is  being  established  to  bring  the  TSE  effort 
under  a  unified  command.  The  COMPMTC  is  now  proposed  to 
"double  hat"  as  AIR  0-6R  in  charge  of  Ranges  S  Targets, 
while  COMNATC  is  to  "double  hat"  as  AIR  0-6T,  in  charge  of 
Test  and  Evaluation.   This  will  put   COMPMTC   in   charge   of 
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some  of  NATC's  resources  and  vice  versa.  This  might  lead  to 
some  problems  but  the  final  details  have  not  been  worked 
yet.  In  any  event,  the  Navy  organization  is  finally 
recognizing  its  inadequacies  in  the  TSF.  area  and  is  working 
towards  a  solution. 
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IV,  FINANCIAL  MANAGEMENT 

In  this  chapter  the  reader  is  introduced  to  both  the 
Navy  Industrial  Fund  (NIF)  and  the  Resource  Management 
System  (RMS)  managerial  concepts.  The  Navy's  RDT&E 
facilities  are  normally  funded  and  financially  managed  by 
one  of  the  two  methods,  either  under  NIF  or  by  institutional 
dollars  under  RMS.  An  examination  of  PMTC  is  then  done  in 
order  to  acquaint  the  reader  with  the  financial  environment 
existing  at  a  T&E  installation. 

PMTC  is  presently  operating  under  both  financial 
methods,  NIF  and  RMS.  This  dual  fiscal  mode  for  PMTC 
resulted  from  the  recent  merger  of  the  Pacific  Missile 
Range,  operating  under  institutionally  funded  RMS,  and  the 
Navy  Missile  Center,  a  NIF  activity.  Continuation  of  the 
existing  two  financial  methods  is  assured  until  FY-77  when 
PMTC  will,  become  a  modified  NIF  organization. 

A.  NAVY  INDUSTRIAL  FUND 


Navy  Industrial  Funding  is  an  accounting  device  that 
introduces  into  the  aanageraent  of  government  industrial  and 
commercial  type  activities  an  integrated  system  of 
accounting  according  to  coaimerical  concepts  including  the 
utilization  of  a  working  capital  account.  Creation  in  1949 
of  an  industrial  fund,  NIF,  within  the  Navy  Department  was 
pursuant  to  the  authority  granted  to  the  Secretary  of 
Defense  under  the  National  Security  Act  Amendment  of  1949 
for  establishment  of  working  capital  funds  in  the  DoD. 

The  needs  for  working  capital  funds  such  as  NIF 
developed  from  Congressional  investigation  into  the 
industrial  facilities  of  the  Military  Departments  during 
WWII.   Studies   made  by  the  various  Congressional  committees 
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pointed  to  a  definite  lack  of  adequate  cost  accounting  and 
the  need  for  some  means  of  accurate,  yet  simple,  cost 
determination.  By  permitting  the  existence  of  working 
capital  funds  Congress  constructed  a  financial  management 
tool  -  "making  those  officals  fully  responsible  for  a  direct 
accounting  for  the  money  they  spend,  the  costing  of  each 
job,  and  the  most  economical  method  of  accomplishing  the 
work"  (Ref.  22). 

Objectives  and  benefits  derived  from  a  NIF  managed 
installation  are  as  follows  in  Reference  22: 

"1.  To  provide  a  more  effective  means  for  cost 
control  over  operations  and  a  more  effective  and 
flexiole  means  for  financing,  budgeting  and 
accounting  for  such  operations. 

2.  To  promote  a  greater  sense  of  responsibility  or 
cost  consciousness  in  customers  of  the  industrial 
fund  installationsi  The  buying  agency  must 
actually  pay  for  the  goods  and  services  furnished 
by  such  installations  based  on  the  availability  of 
its  funds. 

3.  To  permit  industrial  fund  activities  to  more 
effectively  discharge  their  responsibilities  by 
means  of  separate  financing  of  operations  and  less 
cumbersome  budgeting  and  accounting  methods. 

4.  The  managers  of  industrial  fund  activities 
should  be  motivated  to  operate  their  activity  by 
the  most  efficient  means  because  there  will  be 
more  appropriate  indicators  of  efficiency." 

Introduction  of  NIF  to  an  industrial  activity  such  as  a 
shipyard,  an  aircraft  rework  facility,  or  a  research 
installation  is  accomplished  by  the  formalization  of  a 
charter  and  the  establishment  of  a  working  capital  account. 
The  charter  represents  the   approval   of   the   Secretary   of 
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Defense  for  the  activity  to  operate  under  industrial  fund 
procedures.  Additionally,  the  charter  describes  the 
activities  working  capital  requirement,  volume  of  business, 
procedures  and  rates  to  be  used  for  cost  allocation  of 
requested  services. 

A  significant  part  of  the  charter  is  the  providing  of 
initial  working  capital  to  the  new  NIF  activity.  It  provides 
the  cash  to  cover  operating  expenses  and  the  liabilities 
which  are  assumed  initially,  together  with  provision  for  the 
required  investment  in  inventories  of  raw  materials  and 
supplies.  Working  capital  net  amount,  subtracting  such  items 
as  accounts  payable  for  goods  and  services,  and  annual  leave 
of  employees,  is  then  available  as  a  revolving  fund  to 
finaice  the  cost,  of  goods  and  services  produced  for  the 
users  of  the  activity.  Reimbursement  of  the  industrial  fund 
is  accomplished  by  billing  the  appropriation  account  of  the 
using  activity  for  goods  or  services  rendered. 

A  buyer-seller  or  a  contractual  relationship  exists 
between  the  industrial  fund  activity  and  those  organizations 
requiring  its  products  or  services.  Existence  of  a 
buye::-seller  relationship  commences  with  the  submission  of 
an  order  by  the  buyer  for  a  specific  product  or  services  and 
its  acceptance  by  the  selling  activity  at  an  agreed  upon 
price  to  be  paid  by  the  buyer.  Agreed  upon  price  is  at  a 
standard  rate  established  in  the  charter.  The  customer  is 
responsible  for  budgeting  and  budgetary  control  for  the  cost 
of  end-products  and  services  ordered  from  a  NIF  activity. 
Figure  6  illustrates  the  cycle  of  operations  under  NIF 
financing. 

Rate  for  services  and  products  will  vary  among 
facilities  under  NIF  according  to  the  installation's  unique 
overhead  expenses,  use  of  institutional  appropriations,  or 
accounting   method   used   to  distribute  costs.   For  example, 
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data  gathered  on  five  NIF  installations  indicated  a  large 
difference  in  actual  cost  passed  on  to  the  user.  NATC 
Patuxent  River  and  NAPTC  Trenton  have  a  rate  per  hour  to  a 
customer  of  $9.47  and  $9.82,  respectively,  and  a  rate  per 
hour  total  cost  to  the  facility  of  $27.15  for  NATC  and 
$27.49  for  NAPTC.  In  each  case  there  exists  a  difference  of 
approximately  $18  per  hour.  For  NAF  Indianapolis,  NWC  China 
Lake,  and  NMC  Point  Mugu,  both  the  rate  per  hour  total  cost 
and  rate  charge  to  the  customer  were  identical:  $17.58  for 
NAF,  $18.74  for  NWC  and  $18.60  for  NMC  (Ref.   23). 

Dependence  of  an  industrial  fund  activity  on  the 
revenue  it  earns  from  its  customers  to  replenish  the  working 
capital  plus  the  buyer-seller  relationship  are  two 
incentives  which  contribute  to  good  management.  These 
elements  influence  an  industrial  activity  and  its  users  much 
the  same  way  as  customers  and  managers  in  private  business 
are  influenced  by  competition  and  the  necessity  for  economic 
procurement  and  control  of  spending.  A  customer  is  motivated 
to  order  only  that  product  or  service  for  which  he  has  a 
real  need,  since  it  must  be  paid  for  out  of  his  annual 
operating  allotment.  The  contractual  relationship  pressures 
management  of  an  industrial  fund  organization  to  improve 
cost  estimating  and  cost  control.  DoD  Directive  7410.4  of 
September  25,  1972  (Ref.  24)  offers  the  reader  further 
information  on  Industrial  Funds. 

B.   RESOURCE  MANAGEMENT  SYSTEM 

DoD  Directive  7000.1,  as  stated  in  Reference  25, 
reflects  a  change  in  views  within  the  Federal  Government 
concerning  what  constitutes  good  performance  by 
administrators.  Now  performance  is  judged  by 
cost-effectiveness  and  output.  This  is  done  through  the 
measurement  of  management's  use  of  resources  during  specific 
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periods  by  comparing  actual  with  budgeted  expenditures. 
Older  governmental  management  systems  had  different 
objectives  and  were  concerned  only  with  legitimate  use  of 
public  funds.  New  changes  in  performance  criteria  require 
corresponding  changes  in  management  and  information  systems. 

The  Navy  response  to  the  Directive  was  its  Resource 
Management  System.  RMS  is  a  managerial  procedure  which 
improves  the  planning,  control  and  optimal  utilization  of 
scarce  resource  (money,  material  and  manpower)  at  the  lowest 
practical  work  level  of  an  RDTSE"  activity,  identified  as  a 
cost  center.  Participative  management  is  encouraged  within 
the  RDT&E  facility  and  between  the  activity  and  its  Systems 
Command  Headquarter.  Both  the  cost  center  and  activity 
command  management  receive  greater  degree  of  responsibility 
and  a  larger  role  in  decision  making  concerning  what  work 
they  do,  their  budgeting  request,  work  schedule  and 
progress. 

Several  advantages  over  previous  management  systems  are 
accomplished  by  RMS.  Reporting  efforts  under  RMS  show  data 
on  the  same  basis  as  the  budget.  There  is  a  basis  for 
evaluating  performance  of  personnel,  equipment,  material  and 
projects  in  relation  to  cost.  A  history  or  data  base  for 
future  planning  is  created.  Management  decisions  are  made 
more  accurate  and  easier  by  hard  data  (Ref.  24).  The 
improvements  are  a  result  of  cost  center  activity  in 
planning  of  resources,  execution  of  the  budget,  the 
accounting  for  resources,  and  financial  responsibility.  RMS 
combines  the  expanded  managerial  functions  of  cost  centers 
into  a  single  system  in  which  the  functions  interact  with 
each  other. 

For  task  assignments  under  institutional  funding,  RMS 
encourages  a  more  objective  comparison  of  task  output 
against  the  resources   put   into   the   task.    Institutional 
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funding,  sometimes  called  "bucket"  funding,  provides  to  an 
activity  through  its  "Approved  Operating  Budget"  a  source  of 
obligational  authority  by  which  to  operate.  The  "Approved 
Operating  Budget"  is  the  appropriate  Systems  Command 
Headquarter ' s  response  (NAVAIR  for  PMTC)  to  the  organization 
forecast  of  services  rendered  on  task  assignments  from  its 
sponsor.  Figure  7  is  a  graphical  depiction  of  the 
institutional  funding  flow  from  Congress  to  a  cost  center. 

An  activity  is  not  limited  to  performing  services  only 
for  its  sponsor.  It  may  receive  work  request  from  external 
users,  such  as  OPTEVFOR  or  a  NIF  funded  organization.  Work 
performed  on  these  task  assignments  are  on  a  reimbursable 
arrangement.  Customers  are  charged  the  cost  of  the  services 
or  goods  produced  for  them,  which  include  the  direct  cost 
and  possibly  the  allocated  overhead. 

Under  RMS,  all  military  labor,  direct  or  indirect, 
including  duty  is  considered  statistical.  The  label 
"statistical"  means  military  labor  is  not  actually  included 
in  the  total  costs  of  a  cost  center.  It  is  budgeted, 
however,  because  it  is  necessary  to  know  for  managerial  and 
statistical  purposes  the  actual  number  of  military  hours  and 
dollars  used,  and  how  they  are  used.  Statistical  military 
labor  is  often  described  as  unfunded  military  labor  (Ref. 
25)  . 

The  mission  of  the  responsibility  center,  cost  center, 
and  sub-cost  center  determines  the  nature  of  the  workload  to 
be  accomplished.  In  foLraulating  resource  planning 
requirements  to  accomplish  anticipated  aorkload,  work  is 
classified  into  meaningful  categories  of  programs, 
functions,  objectives,  etc.  This  entails  the  identification 
of  specific  objectives,  pLograms,  functions,  tasks,  or 
projects,  establishment  of  appropriate  jou  order  to  collect 
and  identify  costs  incurred,  identification  of  the  resources 
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required,   and   development   of    time    schedules    (fiscal 
quarters)  for  accomplishment  of  effort. 

At  PMTC,  the  RMS  network  consists  of  sub-cost  centers 
(divisions) ,  cost  centers  (departments) ,  and  responsibility 
centers,  which  generally  equate  to  directorate  level 
organization.  The  network  of  cost  centers  facilitates  the 
preparation  of  the  budget,  the  recording  of  accumulated 
expenditures  and,  most  importantly,  the  fixing  of  resource 
responsibility. 

C.   FINANCIAL  MANAGEMENT  AND  FUNDING  PROCEDURES  FOR  PMTC 

PMTC  continues  to  use  the  fiscal  procedures  and 
financial  management  which  existed  prior  to  the  unification 
of  PUR  &  NMC.  PMR,  while  under  the  RMS  managerial  concept, 
received  its  fiscal  procedures  and  funding  guidance  through 
two  mediums:  the  Major  Ranges  and  Test  Facilities  Base 
(MRTFB) ,  as  specified  in  DODDIR  3200.11,  as  well  as  RMS,  as 
promulgated  by  the  Office  of  the  Comptroller,  Navy,  for 
RDTSlf  activities.  MRTFB  prescribes  a  uniform  funding  policy 
that  has  been  implemented  at  all  26  member  activities  of  the 
MRTF3.  The  uniform  funding  policy  conceived  and  executed  by 
the  Defense  Department  states  that  all  direct  costs  of  an 
HBTF3  activity  will  be  paid  by  the  using  activity  or 
organization,  costs  which  are  not  reimbursed  by  the  user, 
such  as  overhead  expenses,  will  be  funded  by  the  DoD 
component  who  is  managing  the  installation,  i.e.,  the  Navy 
for  j?MR. 

A  job  order  cost  accounting  system  is  required  of 
participating  members  by  the  MRTFB  to  support  DoD  uniform 
funding  policy.  Such  a  job  order  system  does  exist  at  PMTC 
for  both  MRTFB  and  NIF  directorates.  Once  a  work  request  is 
accepted  by  PMTC,  an  operation  number   is   assigned   to   it. 
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Then  all  labor  and  material  used  on  the  work  request  is 
recorded  to  that  specified  operation  number.  The  system 
simplifies  the  audit  trail  but  necessitates  a  high  degree  of 
data  recording. 

Directorates  falling  under  HRTFB/RMS  are  those 
comprising  PMR  former  functions  and  personnel  plus  the 
Program  Management  Group  and  the  Threat  Simulation 
Department  from  NHC  and  the  Naval  Air  Station  organization, 
including  the  Aircraft  Maintenance  Department  (AMD) .  AHD 
was  a  separate  department  in  NHC  and  was  operated  under  NIF. 
NIP  managed  directorates  in  PHTC  number  only  three,  with  the 
heart  of  NMC  transfered  to  the  Systems  Evaluation 
Directorate. 

Following  is  a  list  of  the  directorates  and  their 
personnel  level.  Prior  to  consolidation,  PHE  work  force 
consisted  of  556  military  personnel  and  2422  civilians  and 
the  NMC  manpower  complement  was  443  military  and  1,937 
civilians  (Ref .  26) . 
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MRTFB/RMS  directorates  receive  their  operating  dollars 
via  the  Naval  Air  Systems  Command  from  the  Navy  RDTSE 
appropriated  institutional  funds  as  an  RMS  activity.  The 
operating  budget  is  examined  and  approved  by  higher 
authority  within  DoD  and  Congress  under  the  policy  of 
incremental  programming  of  funds.  Incremental  programming 
provides  that  only  those  funds  reguired  for  a  given  fiscal 
year  are  included  in  the  authorization  request.  This 
compels  the  MRTFB  directorates  to  program  the  funding  for 
their  RDT&E  effort  on  an  annual  incremental  basis  as  opposed 
to  the  fully  funded  program  basis  of  the  procurement 
appropriation  (Ref.  27).  Directorates*  approved  operating 
budget,  constituting  a  separate  and  distinct  program  element 
within  the  DoD  budget,  is  the  limit  of  their  obligational 
authority  as  set  by  Congress.  Under  RMS,  obligational 
authority  represents  the  maximum  dollar  value  of  goods  and 
services  which  may  be  ordered  during  the  fiscal  year. 

Institutional  funds  make  up  a  large  part  of  the 
operating  revenues  but  not  all  of  it.   Approximately  a  fifth 
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of  the  dollars  needed  for  day  to  day  operatios  come  from  the 
range's  users  through  direct  cost  charges  to  them  (Ref.  28) 
This  direct  charge  to  users  is  a  recent  phenomenon 
resulting  from  a  Defense  Department  study  commencing  in 
April  1972,  chaired  by  Mr.  George  W.  Bergguist,  Deputy 
Assistant  Secretary  of  Defense  (Systems  Policy  and 
Information) ,  of  the  funding  policy  existing  at  the  26  major 
ranges  and  test  facilities  including,  at  that  time,  PMR. 

One  conclusion  reached  in  Mr.  Bergguist's  investigation 
was  the  necessity  to  improve  cost  control  and  economic 
efficiency  among  the  MRTFB  by  charging  the  users  the  direct 
cost  of  services  or  goods  ordered.  A  direct  consequence  of 
this  study  was  DoD  Directive  3200.11,  dated  June  18,  1974, 
whicn  spells  out  that  "all  DoD  Components  and  other  Federal 
Government  agencies  will  reimburse  the  major  ranges  and  test 
facilities  for  direct  cost".  All  costs  not  identifiable  to 
a  user  are  charged  to  institutional  funds. 

Appendix  A  projects  a  clear  picture  of:  the  growth  of 
direct  cost;  who  the  users  are;  and  past,  present,  and 
forecast  operating  budgets  for  PHR.  Although  PMR  ceases  to 
exist,  the  amount  of  institutional  funds  allotted  to  its 
range  functions  (directorates)  will  continue  as  part  of 
DDT82  financial  planning-  DDTSE  is  the  final  approval 
authority  within  the  Defense  Department  for  budget  requests 
from  any  of  the  national  ranges  and  test  facilities  as 
assigned  by  DODDIR  3200.11. 

Modernization  and  improvement  funds  for  the  RMS 
directorates  come  from  several  appropriation  accounts 
including  the  RDT&E  (N)  institutional  fund  and  Other 
Procurement  Navy  (OPN)  appropriations.  The  PMR  side  of  PMIC 
represents  approximately  a  $150,000,000  investment  in  range 
instrumentation.  An  average  life  expectancy  of  the  major 
range   equipment   is    ten    years,    with    instrumentation 
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obsolescence  occurring  sooner  than  that.  This  would  dictate 
an  annual  investment  of  about  $15,000,000  for  necessary 
replacement  and  update  to  maintain  current  state-of-the-art 
range  instrumentation  systems.  For  the  past  ten  years, 
PMR's  budget  submissions  for  range  modernization  and 
replacement  have  been  cut  in  half.  To  make  up  for  the 
deficiencies  in  appropriations  of  needed  funds  and  to 
fulfill  the  projected  reeds  of  the  Navy  a  $153,980,000  Range 
Instrumentation  Procurement  plan  for  the  years  FY-75  through 
FY-81  was  prepared  (Ref .  28) .  Appendix  A  gives  a  breakdown 
of  this  plan  and  Reference  28  reports  on  the  major  range 
asset  acguistions. 

The  three  directorates  under  NIF  receive  their 
operating  revenue  ultimately  from  the  user  via  the  progress 
payment  method.  Progress  payments  is  the  existing  practice 
used  by  NIF  cost  centers  for  collecting  accrued  costs 
generated  from:  work  requests,  value  of  inventory 
identified  and  reserved  for  the  user's  order,  and  amount  of 
work  in  prpgress.  The  customer  receives  a  monthly  bill  for 
these  costs.  Reimbursement  for  services  or  goods  ordered 
from  the  NIF  cost  center  by  a  user  within  the  Federal 
Government  is  accomplished  by  cross  disbursement  without  the 
use  of  checks.  Eliminating  the  needs  of  using  checks  is 
done  by  charging  the  appropriate  fund  accounts  of  the  user 
activity  and  PMTC  NIF  fund  the  dollar  value  indicated  in  the 
billing.  Collection  of  costs  from  external  customers  to  the 
Federal  Government  is  accomplished  by  checks. 

Internal  billing  between  PMTC's  cost  centers,  both  NIF 
and  MRTFB,  requires  a  specific  set  of  rules  to  ensure 
compliance  with  DoD's  MRTFB  and  Industrial  Fund  regulations. 
MRTF3's  uniform  funding  policy  permits  reimbursement  of 
direct  cost,  only,  and  NIF  allows  the  collection  of  total 
costs.  Direct  work  performed  by  a  NIF  cost  center  for  a  RMS 
customer  cost  center  is  charged  on  the  basis  of  direct  costs 
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plus  overhead.  Work  for  another  PMTC  NIF  cost  center  is 
chargeable  on  the  basis  of  actual  costs,  only,  without 
overhead  application.  -  A  RMS  cost  center  charges  only  direct 
costs  to  a  NIF  cost  center  (Ref .  26) . 

Contrary  to  established  NIF  principles  there  is  not  a 
general  overhead  expense  applied  to  the  total  cost.  This 
slight  deviation  from  NIF  procedures  is  one  of  the  effects 
present  in  PMTC  current  fiscal  operations  that  is  related  to 
its  dual  mode  financial  management.  Instead,  general 
overaead  is  treated  as  an  indirect  overhead,  allocated  on  a 
total  population  basis  and  added  to  a  NIF  directorate  over 
head  expense. 

NIF  directorates  receive  their  funds  for  assets 
acquistion  or  improvement  by  two  means.  The  dollars  are 
either  appropriated  via  a  budget  request  or  become  available 
from  the  industrial  fund.  The  choice  of  which  alternative 
method  to  use  for  equipment  replacement/modernization 
depends  on  the  dollar  value  involved  and/or  initial 
utilization  of  equipment.  Fund  requests  of  less  than  £50,000 
may  be  financed  from  the  industrial  fund  with  recovery  of 
expenditure  coming  from  operating  costs  charged  to  users. 
Costs  of  acquistions  or  improvements  greater  than  $50,000 
will  be  financed  under  appropriated  funds.  Machinery  or 
equipment  authorized  within  a  customer  order  for  fulfilling 
the  work  task  may  be  financed  from  the  industrial  fund  with 
recovery  of  costs  from  the  customer.  Equipment  acquired  in 
this  manner  will  be  disposed  of  in  accordance  with  the 
user's  instruction  (Ref.  24). 

Appendix  A  gives  the  major  users  of  NMC  and  their 
respective  funds  spent  at  the  facility. 

D.  FUTURE  DIRECTION  OF  PMTC  FISCAL  MANAGEMENT 
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For  FY-77  and  on  PMTC  will  operate  under  a  modified  NIF 
concept  called  Reimbursable  Institutional  Funding  (RIF) . 
RIF,  developed  by  NATC  Patuxent  River  Financial  and 
Operating  Cost  Center,  calls  for  institutional  funding  of  an 
installation  overhead  generated  by  its  range  functions,  with 
the  range  services  direct  cost  paid  for  by  the  user.  All 
non-range  cost  centers  will  operate  under  the  normal  NIF 
procedure,  receiving  reimbursement  by  users  for  the  total 
costs  incurred.  Implementation  of  RIF  at  PMTC  will  cause  a 
thorough  examination  of  all  directorates  for  the 
determination  of  cost  centers  which  have  a  range  related 
task  and  those  which  do  not.  The  opportunity  to  have  a  cost 
center  overhead  expense  absorbed  by  institutional  funds  vice 
customers'  dollars  could  have  a  striking  effect  on  the  cost 
estimates  PMTC  will  give  to  customers  in  the  futute.  An 
indication  of  this  is  the  example  stated  earlier  of  five  NIF 
activities  and  the  difference  in  the  rate  per  hour  charged 
to  a  customer  and  that  which  the  installation  experienced. 

By  using  RIF,  PMTC  and  the  Navy  will  be  satisfying  two 
conditions.  The  first  is  the  uniform  funding  plan  advocated 
by  Mr.  Bergguist's  study  group  for  all  26  major  T&S  support 
activities  and  implemented  by  DODDIR  3200.11.  The  second 
condition  being  met  is  the  increased  fiscal  control  and 
motivation  a  NIF  financially  organized  activity  gives  to 
management. 

The  existing  cost  estimate  procedure  used  in  responding 
to  a  customer's  inguiries  is  scheduled  for  improvement.  The 
present  system  calls  for  a  PMTC  Project  Officer,  upon 
receiving  a  proposed  work  request,  to  pass  on  to  the  cost 
centers  concerned  a  list  of  services  or  goods  a  user  seeks 
to  have  filled.  Each  cost  center  forwards  along  its 
proposed  cost  estimate  based  on  the  labor,  instrumentation, 
material,  and  other  items  involved.    The   Project   Officer, 
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assisted   by  a  finanical  analysist,  then  totals  the  bids  and 
presents  PMTC's  proposal  (Ref.  29)  . 

The  current  system  has  several  drawbacks.  A  cost 
center  cost  estimates  for  similar  services  requested  by  two 
different  customers  might  not  be  identical  since  there 
exists  inconsistency  in  who  prepares  the  proposed  costs. 
Likewise,  conflict  could  exists  between  the  Project  Officer 
and  the  cost  center  manager.  A  Project  Officer  is  inclined 
to  present  an  optimistic  price  estimate  on  labor  and 
material  containing  no  margin  for  misfortune  or  overruns  to 
a  customer.  This  is  in  contrast  to  a  cost  center  manager's 
compelling  motivation  to  ensure  full  employment  of  bis 
personnel  and  a  shop  operating  at  full  capacity.  The  cost 
estimates  a  shop  manager  passes  on  for  the  services 
requested  could  reflect  this  view  (Ref.  30) .  The  presence 
of  either  individual's  subconcious  tendency  may  cause  an 
inaccurate  cost  proposal  to  be  presented  to  the  potential 
user,  having  all  concerned  suffer  the  effects. 

PMTC  is  working  towards  comprising  a  "catalogue"  of  all 
the  goods  and  services  that  could  be  offered  by  the 
installation  and  their  stated  costs.  This  proposed  cost 
determination  system  will  necessitate  extensive  research  and 
investigation  into  all  PMTC  services  rendered  and  goods 
produced  (Ref.  29)  .  Initially,  a  determination  of  the 
actual  costs  expended  on  completing  these  functions  in  the 
past  must  be  accomplished.  Once  actual  costs  are  computed, 
an  analysis  should  be  done  on  costs  for  both  validation  of 
cost  averaging  and  to  ensure  the  efficient  use  of  resources 
in  accruing  the  expenses.  This  cost  validation  analysis 
will  be  a  reoccurring  event  to  ensure  cost  proposals  stay  in 
line  with  changing  resource  utilization  efficiency,  labor, 
and  material  rates. 
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v-  TEST  AND  EVALUATION  ASSESSMENT 
A.  MOTIVATION 

Trips  made  to  Point  Hugu  provided  the  authors  with  a 
fairly  comprehensive  impression  of  the  role  field  activities 
play  in  the  TSE  evolution.  Personnel  at  PMTC  were  interested 
in  improving  their  relationship  with  Project  Managers  and, 
consequently,  fostering  a  cooperative  atmosphere.  Personnel 
interviewed  felt  that  they  were  operating  in  a  competitive 
environment  and  that  PMTC  would  have  to  actively  solicit  for 
participation  in  future  weapons  programs.  To  accomplish 
their  objective  they  felt  a  need  to:  first,  know  what  the 
plans  for  future  weapons  systems  are,  second,,  convey  to 
Project  Managers  what  their  capabilities  are  for  meeting 
these  demands.  They  did  not  feel  a  suitable  mechanism 
existed  to  achieve  this  objective. 

The  existence  of  a   competitive   environment   has   also 

been    noted    and   approved   with   restrictions   by   higher 

authority.   The   Chief   of   Naval   Material,   in   NAVMATINST 

5450.27,  stated: 

"...these  activities  have  operated  under  broad  mission 

statements    and    often    overlapping   functional 

assignments  which  have  generated   competition   for 

resources.  While  this  policy  was  acceptable  during 

a  period  of  transition,  continuing  progress  in  the 

establishment   and   strengthening   of  'R&D  centers 

makes  it  desirable  at   this   time   to   modify   and 

restate   policy   so   as   to  focus  more  sharply  the 

resources   and   efforts   of   each   laboratory   and 

center.   It   is   tae  intent  of  this  instruction  to 

strike  a  balance  between,  on  the  one  hand,  a  rigid 

definition    of    functions    that    would   create 

monopolies,  deprive  sponsors  of   flexibility,   and 

lead   to   a   decrease  in  technical  options  and,  on 
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the  other  hand,  a  dispersion  of  capabilities  that 
would  foster  duplication,  decrease  effective 
application  of  resources,  and  promote  competition 
for  programs  and  resources.  Flexibility  and 
dynamic  operation  are  recognized  as  keys  to 
success  in  research  and  development  operations. 
Thus  the  policies  enunciated  herein  are  designed 
to  encourage  creativity,-  to  provide  alternative 
technical  approaches,  to  control  competition,  and 
to  ensure  coordination  of  in-house  efforts." 

In  order  to  obtain  a  clearer  idea  of  the  relationship 
existing  between  the  user  and  the  facilities,  it  was  decided 
to  obtain  the  opinions  of  the  Project  Managers.  Road  trips 
by  PMTC  personnel  to  Washington  D.  C.  indicated  a  wide 
variety  of  viewpoints  existed  concerning  this  relationship 
(Kef.  31).  The  data  obtained  was  considered  insufficient  to 
form  any  definite  conclusions.  The  best  method  for 
obtaining  additional  information  would  be  through  personal 
interviews  with  the  Washington  based  Project  Managers. 
Although  desirable,  this  method  was  discounted  due  to 
funding  and  time  constraints.  The  next  best  alternative  was 
to  conduct  a  nail  survey. 

Formulation  of  the  questionnaire  was  predicated  on 
learning  the  views  and  opinions  of  Project  Managers  in 
certiin  key  areas  of  T6E.  Through  background  investigation, 
research  trips,  and  interviews  conducted  relative  to  the  Pils 
and  their  interaction  with  the  facilities,  there  surfaced 
five  managerial  functions  that  needed  further  exploration. 
Little  was  understood  oi  the  criteria  used  by  PMs  in  their 
selection  of  a  T&E  facility;  what  were  the  factors  and  their 
order  of  importance?  The  latitude  a  Project  Manager  has  in 
selecting  a  T&E  facility  was  not  known.  It  was  felt  that 
constraints  such  as  technical  conditions,  project  urgency, 
or   DCP   requirements  might  dictate  which  facility  a  PH  must 
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use.  The  method  used  by  a  Project  Manager  to  familiarize 
himself  with  the  different  installations  was  not  known.  Does 
he  rely  on  past  experience,  other  PMs,  publications,  liaison 
offices,  a  staff  within  the  Systems  Command,  or  a 
combination  of  these  as  his  information  source?  Since 
competition  does  exist  between  T&E  facilities,  it  was 
thought  important  to  measure  the  Pil's  feelings  towards  this 
competitive  environment.  Furthermore,  what  were  their 
opinions  on  the  quality  of  services  provided? 

Once  the  categories  for  PM's  responses  were  determine!, 
formulation  of  the  statements  to  explore  those  areas  was 
begun.  Wording  of  individual  statements  was  aided  through 
discussions  held  with  individuals  having  T&E  experience. 
Construction  and  validation  of  the  statements  was  an 
involved  process  requiring  the  selection  of  response  format 
and  data  analysis  to  be  used,  and  location  of  a  test 
sampling  group  to  ensure  question  content  and  validity.  A 
five  choice  response  format  was  chosen  as  a  reasonable 
solution  to  both  the  response  format  and  data  analysis 
requirements.  This  response  format  provided  ease  of  data 
reduction  and  adaptability  to  the  Statistical  Package  for 
the  Social  Sciences  (SPSS)  on  file  at  the  Naval  Postgraduate 
School  computer  center.  It  also  permitted  the  respondent  a 
degree  of  flexibility  in  answering  the  questions. 

After  each  category  was  thoroughly  covered  by  its 
statements,  an  initial  version  of  the  questionnaire  was 
prepared.  Personnel  with  prior  TSE  experience  were  located 
and  used  as  the  test  sampling  group.  Objectives  of  the 
questionnaire  were  explained  to  the  individuals.  The  group 
was  asked  not  only  to  complete  the  questionnaire  but  to 
communicate  their  comments  concerning  the  composition  and 
validity  of  each  statement  to  the  authors,  either  on  the 
questionnaire  or  in  person.  Responses  from  the  sample  group 
were   used   to   modify   and  improve  the  questionnaire  before 
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mailing.  Although  the  questionnaire  was  principally 
designed  and  directed  to  the  Project  Managers,  it  was  felt 
some  of  the  same  issues  were  applicable  to  the  OPTEVFOR 
community.  Hence,  the  questionnaire  was  sent  to  VX-1  and 
VX-5. 

B.   QUESTIONNAIRE 

This  section  presents  the  objective,  result,  and  the 
conclusion  reached  for  each  question.  Responses  received 
from  the  questionnaire  were  examined  and  transfered  to  data 
cards  for  tabulation  by  the  SPSS  package.  Results  were  then 
printed  out  in  a  format  similar  to  that  used  in  the  SPSS 
package.  Appendix  B  contains  the  questionnaire  and  input 
data.  Abbreviations  utilized  are  as  follows:  S. A. -strongly 
agree,  A. -agree,  N.O.-no  opinion,  D.-disagree/  S. D. -strongly 
disagree . 

P.u§s.t:i°Il  ll  f:ore  than  one  Test  and  Evaluation  (T&E)  facility 
was  capable  of  providing  the  similar  services  needed  for  my 
pro jsct. 

1.  Objective:  Desired  to  establish  the  existence 
of  more  than  one  T&E  facility  which  had  similar 
environment  (land,  sea,  or  air),  technical  support,  and 
physical  plant  to  conduct  the  T&E  required  on  a 
project.  An  implied  assumption  was  made  that  no  two 
facilities  are  identical  in  capabilities;  but  when  an 
installation's  functions  are  presented  in  the  aggregate 
the  summation  of  those  capabilities  are  approximately 
equal  to  another  installation  capability  total.  This 
information  could  have  been  partially  retrieved  from  a 
library  research  of  the  DoD's  ranges  and  test 
facilities.  A  search  of  that  type  however,  would  not 
have  indicated  the  perception  held  by  Project  Managers 
and  Project  Officers  towards  the  existence  of  more  than 
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one  T&E    facility  capable  of  meeting  their   needs.    One 

of   the   factors   needed  to   show   the   existence  of  a 

competitive  environment  was   duplicate  capability   at 
various  installations. 

2.  Results: 


Responses  S.A. 

PM'S     20  19.0% 

VX'S     20  10.0% 

ALL      40  14. 6=6 


PM'S 
VX'S 
ALL 


MEAN 
2.952 
2.550 
2.756 


A. 

N.O. 

D. 

S.D. 

33.3% 

0 

28.6% 

19.0% 

60.0% 

5.0% 

15.0% 

10.0% 

4  6.3% 

2.4% 

22.0% 

14.6% 

MEDIAN 

2.429 

2.167 

2.263 

3.  Conclusions;  Most  managers  surveyed  agree  that 
duplicate  capability  exists  at  many  installations. 
Most  noticeable  is  the  almost  even  split  among  the 
Project  Managers  in  Washington  towards  this  statement. 
Since  facilities  with  comparable  functions  are  present, 
it  is  reasonable  to  assume  the  occurrance  of 
competition  between  activities. 

Question  2l  Tne  technical  constraints  of  my  project   limited 
selection  to  one  TSE  facility 

1.  Objective:  It  was  believed  that  the  technical 
requirements  of  a  program's  T&E  could  be  of  such  an 
unique  nature  as  to  restrict  a  Project  Manager  to  a 
single  facility.  There  are  several  weapon  systems  that 
do  require  special  TSE  functions  and  would  confine  the 
selection  process  to  only  one  activity.  The  relative 
size  of  the  two  groups,  technically  restrictive  and  non 
restrictive,  was  unknown.  Additionally,  the  statement 
is  again  asking  the  individual  to  judge  if  duplicate 
TSE  fcicilities  exist. 
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PM<S 
VX'S 


:.  Results: 

Responses  S.A. 

20  15. OX 

20  10.0* 


ALL      40        12.5% 


PM»S 
VX«  S 
ALL 


MEAN 
3.000 
3.300 
3.  150 


A.                I 

S.O. 

D. 

S.D. 

30.0% 

5.0% 

40.0% 

10.0% 

20. 0* 

5.  Of. 

60.0% 

5.0% 

25.0% 

5.0% 

50.0% 

7.5% 

MEDIAN 

3.500 

3.750 

3.650 

3.  Conclusion:  Responses  from  Project  Managers 
are  the  significant  ones  in  this  statement.  Project 
Managers  are  concerned  with  a  weapon  system  RSD  and 
developmental  TSE  in  constrast  to  OPTEVOR'S  function  of 
operational  TSE.  RSD  and  developmental  TSE  contain  the 
area  of  greater  technical  constraints.  Size  of  the  two 
groups,  technically  restrictive  and  non  restrictive,  as 
viewed  by  Washington  is  approximately  equal.  A 
sizeable  numbers  of  projects  are  compelled  to  use  the 
one  facility  possessing  those  unique  characteristics 
needed  for  the  weapon  system  TSE.  Good  correlation 
exists  between  questions  1  and  2  relative  to  duplicate 
TSE  facilities,  i.e.,  45%  of  the  Project  Managers 
believed  they  are  technically  constrained  and  48,? 
believed  duplicate  ISE  facilities  do  not  exist  for 
their  project. 

Question   ^1   Tne  urgency  of  my  project  limited  selection  to 
one  TSE  facility. 

1:  Objective:  Two  factors  were  considered  in  this 
statement,  time  and  existing  capabilities  at  TSE 
installations.  The  two  elements  are  directly  related 
and  dependent  on  the  other.  The  longer  the  lead  time  a 
project   has,   the   greater   is   the   possibility   that 
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additional  facilities  could  adapt  to  the  T&E 
specification  requirements;  and  conversely  for  a  short 
lead  time.  As  with  the  objectives  for  question  2,  the 
relative  size  of  these  two  groups,  short  lead  time  or 
long,  was  unknown. 

2.  Results: 


Responses 

S.A 

• 

A. 

N.O. 

D. 

S.D. 

PM'S 

20 

5. 

0% 

30.0% 

20.0% 

4  0.0% 

5.0% 

VX«  S 

20 

0 

5.0% 

25.0% 

60.0% 

10.0% 

ALL 

40 

2. 

5% 

17.5% 

22.5% 

50.0% 

7.5% 

MEAN 

MEDIAb 

I 

PM'S 

3. 

100 

3.250 

vx»s 

3. 

750 

3.833 

ALL 

3. 

425 

3.650 

.  3:  Conclusion:  Urgency  plays  a  signif iciaiit  role 
at  the  Project  Manager  level  and  is  almost  nonexistent 
in  OPTEVFOR.  Urgency  could  be  another  element  affecting 
PM's  perception  of  duplicate  capability  among  T&E 
installations. 

Question,  la   Current  publications  provide  a  useful  source  of 
information  on  the  capabilities  of  T&E  facilities. 

1:  Objective:  Very  little  is  understood  of  the 
technique  used  by  Project  Managers  to  acquaint 
themselves  with  the  various  T&E  facilities.  There  are 
numerous  publicat ions,  distributed  both  by  the 
installations  and  the  Services,  that  describe  the 
mission,  functions,  and  existing  technical  assets  of 
the  T&E  facilities.  It  was  felt  that  these  publications 
either  do  not  meet  the  information  needs  of  the  Project 
Mangers  or  are  unfamiliar  to  them. 

2.  Results: 
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es   S. A. 

A. 

N.O. 

D. 

S.D. 

0 

42.9% 

23.8% 

23.8% 

9.5% 

0 

25.0% 

10.0% 

40.0% 

25.0^ 

0 

3  4.1% 

17.1% 

31.7% 

17.1% 

MEAN 

MEDIA* 

f 

3.000 

2.800 

3.650 

3.875 

3.317 

3.429 

PM'S  21 
VX'S  20 
ALL      41 

PM'S 
VX'S 

ALL 

3.  Conclusion:  Printed  information  on 
installations  and  services  they  can  offer  needs  to  be 
improved.  Current  publications  are  not  providing  a 
useful  source  of  information  to  OPTEVFOR  and  are 
disregarded  by  more  than  half  of  the  PMs.  It  appears 
that  an  improved  flow  of  information  to  the  users  of 
T&E  facilities  at  an  economical  cost  is  possible 
through  better  constructed  and  formulated  publications. 

Question   5x   Other  Pro ject  Managers  provide  a  useful  source 
of  information  on  the  capabilities  of  T&E  facilities. 

1.  Objective:  In  the  investigation  of  the  method 
used  by  Project  Managers  in  gaining  a  knowledge  of  an 
installation's  functions  and  ability,  it  was  suspected 
that  other  individuals  play  a  role.  A  key  person  might 
be  another  Project  Manager.  Possibly  his  predecessor, 
in  the  pass  down,  left  either  written  or  verbal 
information  as  to  a  facility's  capabilities. 
Additionally,  existing  Project  Managers,  acting  as  a 
corporate  memory,  could  be  a  source  of  knowledge  and  a 
readily  available  group  to  answer  questions. 
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2.  Results: 

Responses 
PM'S     21 
VX'S     20 
ALL      41 


es   5.  A 

>  • 

A. 

N.O. 

D. 

S.D. 

4. 

8% 

6  1.9% 

14.3% 

14.3% 

4„8X 

30. 

0% 

35.5% 

10.0% 

25.0% 

0 

17. 

155 

48.8% 

12.2% 

19.5% 

2.4% 

MEAN 

MEDIAN 

f 

2.524 

2.231 

2.300 

2.071 

2.415 

2.175 

PM  '  S 
VX«  S 
ALL 

3.  Conclusion:  An  important  reservoir  of  knowledge 
and  information  is  an  individual's  contemporaries.  The 
reputation  and  past  associations  a  facility  has  among 
this  group  will  .  affect  future  relations.  Tnis 
"informal"  information  link  can  provide  much  valuable 
information  which  would  require  volumes  of  technical 
manuals  to  cover.  There  is,  however,  the  danger  of 
loss  of  objectivity  in  the  utilization  of  such  a 
source.  One  especially  good  or  bad  incident  may 
greatly  color  an  individual's  attitude  toward  an 
installation.  There  also  exists  the  danger  of  not 
completely  weighing  all  the  factors  but  merely  going 
with  the  "current  favorites". 

Question  6j_  Liaisons  from  T&E  facilities  provide  a  usefal 
source  of  information  on  the  capabilities  of  their 
facilities. 

1.  Objective:  Most  TSE  facilities  maintain  a 
Washington  liaison  office.  The  office  forms  a 
communication  channel  between  the  Systems  Command 
Headquarter  and  the  field  installation.  It  would  appear 
that  these  liaison  offices  should  be  a  center  of 
available  information  and  a  sou.:cc  capable  of  answering 
a  Project  Manager's  inquiries.  An  indication  oi  their 
effectiveness   as  a  useful  source  of  information  should 
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es   S.I 

L  . 

A. 

N.O. 

D. 

S.D. 

19. 

OS 

42.9% 

9.  5% 

23.8% 

4.8% 

20. 

0% 

6O.OF0 

5.0% 

1  5 .  0  % 

0 

19. 

,5% 

51.2% 

7.3% 

19.5% 

2.4^ 

MEAN 

MEDIAN 

2.524 

2.222 

2.150 

2.000 

2.341 

2.095 

have  come  from  the  responses  given. 
2.  Results: 


PM'S  21 
VX'S  20 
ALL      41 

PM'S 
VX'S 
ALL 

3.  Conclusion:  Liaison  between  the  T8E  activity 
and  OPTEVFOR  appears  to  be  strong  and  effective.  There 
is  adequate  communication  between  Washington  and  the 
TSE  installations,  but  not  totally  effective.  A  large 
number  of  Project  Managers  are  not  being  reached,  as 
indicated  by  the  38%  negative  response  from  this  group. 
A  liaison  office  which  can  effectively  present  current 
information  on  its  installation  capabilities  and 
technical  expertise  to  the  PMs  could  be  a  valuable, 
asset  to  a  decision  maker  during  his  selection  process. 

Question  7j_  A  Project  Manager's  previous  ^ssic^nnierit  provides 
a  valuable  source  on  information  on  the  capabilities  of  T&2 
f aciiiies. 

1.  Objective:  It  was  thought  that  a  Project 
Manager  might  already  have  a  well  conceived  impression 
of  existing  capabilities  at  TSE  installations.  These 
impressions  are  based  upon  previous  experiences  gained 
from  T&E  related  billets  or  operational  duties. 
Question  36  addresses  previous  T&E  experiences. 
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A. 

N.O. 

D. 

S.D. 

28.6% 

9.5% 

2  3.8% 

0 

30.0% 

10.0% 

10.0% 

15.0% 

29.3% 

9.8% 

17.  1% 

7.3% 

MEDIAN 

1.917 

2.000 

1.958 

2.  Results: 

Responses  S.A. 

PIT'S     21  ..  38.1% 

VX'S     20  35.0% 

ALL      U1  36.6% 

MEAN 
PM'S  2.190 

VX'S  2.400 

ALL  2.293 

3.  Conclusion:  As  stated  in  the  conclusion  of 
Question  5,  reputation  and  past  associations  between  an 
individual  and  a  facility  will  form  the  basis  for 
future  relations.  A  significant  number  of  individuals, 
2/3  of  the  respondents,  have  formed  opinions  of  various 
T&E  inscallations  prior  to  assuming  their  current 
position.  A  decision  maker's  evaluation  of  an 
installation's  capabilities  and  expertise  based 
partially  on  past  memory  contains  an  inherent  danger. 
Past  memory  can  be  heavily  influenced  by  one 
siginificant  happening,  good  or  bad,  which  occured 
during  an  individual's  association  with  the  activity 
and  obscure  the  existing  capability  level  of  the 
facility. 

£u.ss_tion  &1  Staff  assistance  within   a   Systems.   Command   is 
useful  in  determining  the  capabilities  of  T&E  facilities. 

1,  Objective:  Determine  if  a  Project  Hanager 
receives  assistance  from  his  Systems  Command  on 
facility  selection.  It  was  thought  that  the  Systeas 
Command  should  perform  a  staff  function  of  aiding  a  PK 
during  his  analysis  of  T&E  requirements  and 
installations  capable  of  satisfying  them.  ihe  Systems 
Command  staff  should  act  as  an  independent  cent  i  ,ili7.>",d 
information  source  on  T&E  facilities  and,  consequently , 
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A. 

N.O. 

D. 

S.D. 

55.0% 

10.0% 

25.0% 

5.0% 

50.0% 

35.0% 

10.0% 

5.0% 

52.5% 

22.5% 

17.5% 

5.0% 

MEDIA* 

f 

2.318 

2.500 

2.405 

supplement   a   PM's   small   staff   with    knowledgealbe 
personnel  during  the  selection  of  a  T&E  facility. 

2.  Results: 

Responses  S.A. 
PM'S     20         5.0% 
VX'S     20         0 
ALL      40         2.5% 

MEAN 
PM'S  2.700 

VX'S  2/700 

ALL  2.700 

3.  Conclusion:  Some  staff  'assistance  is  occuring 
at  both  the  OPTEVFOR  and  Systems  Command  levels. 
Improved  staff  assistance  might  reduce  the  reliance  on 
less  than  totally  objective  information  presently  being 
used  by  an  individual.  Information  received  from 
liaison  offices,  coretemporaries,  and  past  assignments 
can  fall  into  this  category  of  "less  than  totally 
objective". 

£u£stion   92,   There   is  a  significant  difference  in  costs  to 
projects  from  T&E  facilities  offering  comparable  services. 

1.  Objective:  This  stateroeat  and  the  next  was 
prepared  to  help  investigate  the  role  played  by  costs 
in  selecting  a  T&E  activity.  From  research  done  into 
the  fiscal  area  of  T&E,  it  is  known  that  various 
installations  can  offer  significantly  different  cost 
proposals.  (The  uniform  funding  policy  established  for 
the  MRTFB  does  not  define  a  uniform  cost  rate  to  be 
used.  Uniform  funding  policy  indicates  who  pays  the 
overhead,  and  leaves  the  determination  of  direct  labor 
and  material  rates  to  the  individual  activity.) 
Response   to   the  question  would  indicate  the  awareness 
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of  Project  Managers  and  Project  Officers  to  costs  and 
indirectly  the  importance  of  costs  in  their  selection. 
Queston  16  produces  futher  information  on  an 
individual's  concern  for  cost  proposals.  Comparable 
services  was  used  to  emphasis  that  the  the  variances  in 
costs  were  for  similar  work  requests. 

2.  Results: 

Responses  S.A.      A.  N.O.  D.  S«D. 

PM'S     21  19.0%  33.3%  38.1%  9.5%  0 

VX'S     20  30.0%  30.0%  23.0%  15.0^  0 

ALL      41  24.4%  31.7%  31.755  12.2%     0 

MEAN  MEDIAN 

PM'S  2.381  2.429 

VX'S  2.250  2.167 

ALL         '  2.317  2.308 

3.  Conclusion:  A  majority  of  those  responding 
indicate  their  acknowledgement  of  substantial  cost 
differences.  What  is  interesting  is  the  large 
percentage  of  no  opinion  registered  to  this  statement. 
It  is  hard  to  conceive  of  stimulating  a  blank  comment, 
no  opinion,  towards  the  question  unless  one  is 
unconcerned  with  costs. 

£ii£§tion  10.JL  Different  funding  techniques  (NIF/RMS ,  etc)  used 
by  TSE  facilities  tend  to  mask  the  actual  T&E  cost  of  a 
project. 

1.  Objective:  As  discussed  in  Chapter  4  on 
financial  management,  costs  paid  by  a  PM  could  be 
either  direct  or  total  depending  on  what  funding 
system,  NIF  or  MRTFE,  the  installation  is  under.  The 
funding  method  in  relation  to  costs  projected  to  a  user 
is  immaterial  to  Services  and  utlimately  DoD,  as  the 
Defense   Department   roust  budget  and  pay  for  all  costs. 


A. 

N.O. 

D. 

S.D 

33.3% 

23.8% 

4.8% 

0 

25.0% 

35.0% 

0 

0 

29.0% 

29.3% 

2.14% 

0 

MEDtAi< 

I 

1.857 

1.900 

1.875 

If  a  Project  Manager  was  not  aware  of  these  facts,  he 
could  be  swayed  to  select  a  facility  that  on  the 
surface  was  projecting  the  least  cost,  but  has  the 
greatest  total  costs.  The  reader  is  referred  back  to 
Chapter  4  Section  1  and  the  example  of  the  five  NIF 
activities'  rate  per  hour  to  customers  and  total  costs. 

2.  Results: 

Responses  S.A. 

PM'S     21  38. U 

VX'S     20  40.0% 

ALL      41  39.0% 

MEAN 
PM'S  1.952 

VX'S  1.9  50 

ALL  1.951 

3.  Conclusion:  Most  individuals  are  aware  of  the 
effects  certain  funding  techniques  have  on  masking  the 
actual  total  costs  incurred  on  a  project  to  the  DcD. 
There  appears  to  be  some  inconsistency  between  the 
responses  in  this  question  and  the  last  one  on 
significant  cost  differences,  especially  from  Project 
Managers.  One  would  expect  a  higher  no  opinion- 
response  on  a  more  sosphiscated  statement  such  as  this 
one,  but  the  opposite  happened.  It  is  easier  to 
explain  a  lack  of  knowledge  in  a  specified  area  then  to 
justify  a  lack  of  concern. 

O.^£st.ion  ill  Once  a  T£E  facility  has  been  chosen  for  a 
project  a  project  manager  has  no  latitude  in  altering  the 
selection. 

1.  Objective:  Once  a  selection  was  made 
designating  a  facility  to  perform  certain  TiE 
functions,   it  was  not  known  if  this  selection  could  be 
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altered.  Futhertnore  a  doubt  persisted  as  to  the 
existence  of  Project  Managers  and  Project  Officers' 
authority  to  execute  such  altering.  It  was  assumed 
that  the  change  in  TGE  facilities  was  due  to  cause. 

2.  Results: 


Responses 

;   S.  A. 

A. 

N.O. 

D. 

S.D. 

PM«S 

20 

10. 

0% 

25.0% 

5.0% 

55.0% 

5.0% 

VX'S 

20 

5. 

.0% 

10.0% 

40.0% 

40.0% 

5.0% 

ALL 

40 

7. 

5% 

17.5% 

22.5% 

47.5% 

5c  0% 

MEAN 

MEDIAN 

PM«S 

3. 

200 

3.682 

VX'S 

3. 

300 

3.375 

ALL 

3. 

250 

3.553 

3.  Conclusion:  Selection  of  a  T&E  facility  can  be 
altered  and  a  Project  Manager  does  have  the  authority 
to  execute  the  change.  Responses  from  OPTEVFOR  present 
a  mixed  picture  as  to  the  latitude  of  a  Project  Officer 
in  changing  TSE  facilities.  Approximately  half  believe 
they  do  possess  the  authority  and  the  other  half  does 
not  know,  as  represented  in  the  no  opinion  percentage. 

Question  Ylz_  As  regards  to  facilities  offering  comparable 
services,  there  is  a  significant  difference  in  the 
iB.iL2£!sLii:2I*.  ^^§£.^^£K  provided  to  a  project  manager  to 
effectively  monitor  the  progress  of  his  project. 

1.  Objective:  There  presently  exists  varying  forms 
of  management  information  systems  at.  T&E  installations. 
A  management  information  system  should  feed  useful  and 
timely  data  not  only  to  those  on  the  station  but  also 
to  a  PM,  who  is  ultimately  responsible  for  his  project. 
It  was  not  known  if  wide  differences  of  quality  existed 
in  management  information  systems.  The  existence  of  a 
better  information  system  which  gives  a  Program  Manager 
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es   S.A. 

A. 

N.O. 

D. 

S.D. 

25:0% 

30.0% 

40.0% 

5.0% 

0 

15.0% 

25.0% 

40.0% 

15.0% 

5.0% 

20.0% 

27.5% 

40.0% 

10.0% 

2.5% 

MEAN 

MEDIAN 

2.250 

2.333 

2.700 

2.750 

2.475 

2.56  3 

improved  control  over  his  program,  night  persuade  hi  in 
to  select  and  work  with  that  facility.  This  could  be 
especially  important  to  a  manager  from  the  Washington 
area  working  with  a  distant  facility. 

2.  Results: 


PKrS  20 
VX'S  20 
ALL      40 


PM'S 
VX'S 
ALL 

3.  Conclusion:  The  objective  was  not  met  due  to 
poor  formulation  of  the  question.  Some  believe  good 
information  systems  do  exist.  Others  did  not  know  how 
to  interpret  the  statement,  as  indicated  by  the  large 
no  opinion  response. 

^!i§Sti°J}  Hi  There  is  a  significant  difference  in  the 
technical  expertise  of  r&E  facilities  offering  comparable 
services . 

1.  Objective:  T&£  activities  do  offer  similar 
functions  and  capabilities,  but  this  does  not  give  a 
complete  picture  of  their  ability  to  perform  T&E. 
Personnel  experience  and  reputation  play  an  important 
role  in  the  successful  completion  or  a  program.  Do 
differences  in  technical  expertise  and  reputation  exist 
between  two  duplicate  capability  installations? 
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2.  Results 


es   S.A. 

A. 

N.O. 

P- 

S.D 

30. 

,0% 

45.0% 

10.0% 

15.0ft 

0 

20. 

.0% 

20.0% 

30.0% 

30.0% 

0 

25. 

,0% 

32.5% 

20.0% 

22.5% 

0 

MEAN 

MEDIAN 

2.  100 

1.944 

2.700 

2.833 

2.400 

2.269 

PM'S  20 
VX'S  20 
ALL      40 

PM'S 
VX's 
ALL 

3.  Conclusion:  At  the  Washington  level  a 
perception  of  different  technical  expertise  strongly 
exists.  In  the  Binds  of  the  Project  Managers,  some  T&E 
facilities  have  superior  personnel,  and  to  take  the 
results  a  step  futher,  superior  quality  of 
capabilities.  QPTEVF0R  projects  a  mixed  feeling  and  no 
futher  conclusion  can  be  drawn. 

2ii£5ii21i  li£:  As  regards  to  facilities  offering 
comparable  services,  there  is  a  significant  difference 
in  the  ease  of  obtaining  non-routine  information  on  the 
progress  of  a  project. 

1.  Objective:  Apart  from  the  periodic  reporting  on 
a  project's  status  the  Project  Manager  often  has  need 
for  additional  information  not  contained  in  these 
reports.  Personal  interviews  have  indicated  some 
difficulty  in  obtaining  non-routine  information.  In 
particuliar,  some  activities  have  more  than  one  contact 
point  for  information  concerning  a  project.  This 
question  was  designed  to  measure  the  ease  of  obtaining 
non-routine  information.  Results  could  be  useful  in 
determining  the  overall  coordination  and  communicative 
feedback  existing  within  the  activities. 
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A. 

N.C 

). 

D. 

S.D. 

25.0% 

30. 

0% 

20. 

0% 

0 

40.0% 

35. 

0% 

0 

0 

32.5% 

32. 

5% 

10. 

0% 

0 

MEDIAN 

2.5 

2.125 

2.269 

2.  Results: 

Responses  S.A. 

PM«S     20  25.0% 

VX»S     20  25.0^ 

ALL      20  25.0% 

MEAN 
PM»S  2.45 

VX'S  2.1 

ALL  2.275 

3.  Conclusions:  This  -question  appears  to  have 
raised  a  valid  point,  since  only  ten  percent  of  the 
respondents  indicated  disagreement.  Interestingly 
enough,  none  of  the  VX  respondents  disagreed  which  may 
raise  doubt  az  to  facilities'  overall  cooperation  with 
operational  T&E  units.  Perhaps  the  lower  rank  of  VX 
Project  Officers  in  comparison  with  Project  Managers 
might  have  an  effect  on  the  ease  of  obtaining 
non-routine  information.  Results  from  this  question 
indicate  that  some  facilities  have  done  a  better  job  of 
providiing  information  services  to  customers  than 
others . 
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£u££ti2I}  15:.  Funding  level  for  T&E  facilities'  modernization 
is  adequate  to  support  my  project's  needs. 

1.  Objective:  Recent  cuts  in  the  Defense  budget 
have  resulted  in  the  curtailment  of  modernization  and 
expansion  of  TCE  facilities.  The  effects  of  these 
cutbacks  may  have  already  begun  to  decrease  TGE 
facilities'  effectiveness  in  meeting  a  user's  needs. 
This  question  was  designed  to  evaluate  the  degree  to 
which  existing  projects  have  been  affected  by  a 
reduction  in  Defense  spending. 

2.  Results: 

Responses  S.A. 
PM'S     19         5.3% 
VX*S     20  0 

ALL      39         2.6% 


PM'S 
VX'S 
ALL 


MEAN 
2.842 
3.2 
3.026 


A. 

N.O. 

D. 

S.D. 

36.8% 

36.8% 

10.5% 

10.5CS 

25.0% 

4  0.0* 

25.0?3 

10.055 

3  0.8% 

3  8.5!* 

17.94 

10.32 

MEDIAN 

2.714 

3.125 

2.933 

3.  Conclusions:  Overall,  opinions  on  this  question 
were  evenly  spread  with  VX  respondents  indicating  so.^e 
dissatisfaction  with  project  support.  Since  there  is  no 
previous  data  to  indicate  how  Project  Managers/Officers 
felt  in  the  past  concerning  funding  adequacy  no  valid 
conclusions  could  be  made  as  to  whether  cutbacks  were 
significantly  affecting  program  support. 
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2u£§tion  Iki.   Rank  each  of  the  following  factors  with  respect 
to  its  importance  in  the  facility  selection  process. 

Cost  Estimates 

Personnel  Expertise 

Technical  Capability 

Logistics 

Program  Cooperation 

Test  Scheduling 

Others  (specify) 

1.  Objective:  In  selecting  a  TSE  facility  offering 
comparable  services  for  his  project  a  Project  Manager 
presumeably  considers  the  comparability  between 
facilities  for  the  identified  parameters.  Personal 
interviews  indicate  that  a  facility  may  tend  to 
understate  their  cost  estimates  in  order  to  attract 
additional  business.  In  any  event  there  may  exist  a 
wide  spread  difference  between  estimated  and  actual 
costs  which  would  tend  to  diminish  the  credibility  of 
cost  estimates  and  consequently  decrease  its  importance 
in  the  facility  selection  process.  The  logistics 
parameter  is  aimed  at  identifying  to  what  extent  a 
facility's  geographical  location  and  support  services 
take  in  the  selection  consideration.  The  other 
parameters  are  indications  of  the  quality  of  services 
provided  which  may  vary  extensively  between  facilities 
The  objective  of  this  question  was  to  determine  which 
parameters  are  most  important  to  a  Project  Manager  in 
considering  facility  selection. 
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(1  =  Host  Important) 
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:tion    I 
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1       2 
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26       5 
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2       1 
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3       4 

10 

8 

7 

6 

1 

3.872 

0       4 

5 

6 

10 

10 

0 

4.436 

0       0 

0 

1 

0 

0 

tt 

! 

5.500 

2.  Results: 
FREQUENCY  OF  RANKING 

Selection  Parameter 

Cost  Estimates 

Personnel  Expertise 

Teen.  Capability 

Logistics 

Program  Cooperation 

Test  Scheduling 

Others 

3.  Conclusions:  Respondents  generally  agreed  that 
technical  capability  and  personnel  expertise  were  of 
primary  importance.  Cost  estimates  narrowly  outranked 
program  cooperation  for  the  third  highest  ranking. 
Apparently  logistics  and  test  scheduling  are  of  little 
importance  compared  to  the  other  parameters.  Two 
respondents  provided  additional  parameters:  one 
••unspecified",  the  other  ranked  "ability  to  simulate 
operational  environment"  fourth. 

Question  JZl  Rank  the  following  T£E  facilities  "in  their 
overall  effectiveness  in  providing  satisfactory  services  to 
Project  Managers. 


1.  Most  Effective  -  - 


8.  Least  Effective 


A.  Naval  Weapon  Center,  China  Lake 

B.  Pacific  Missile  Test  Center,  Point  Mugu 

C.  Naval  Air  Test  Center,  Patuxent  River 

D.  Armament  Development  and  Test  Center,  Eglin  AFE 

E.  White  Sands  Missile  Range 
f.  U.S.  Army  Proving  Grounds 

G,  Atlantic  Fleet  Weapons  Range 

H.  Air  Force  Eastern  Test  Range,  Patrick  AFB 
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1.  Objective;  This  guestion  was  designed  to 
determine  the  image  different  T&E  facilities  project  to 
potential  users.  The  choice  was  limited  to  facilities 
capable  of  providing  services  to  airborne  weapon 
systems  and  hopefully  the  personnel  surveyed  would  have 
some  familiarity  with  or  impressions  of  the 
effectiveness  of  the  respective  facilities.  This 
question  was  not  designed  to  determine  a  facility's 
actual  effectiveness  but  merely  to  ascertain  an  idea  of 
the  reputation  enjoyed  by  these  facilities. 

2.  Results: 

FREQUENCY  OF  EFFECTIVENESS  RANKING 
Relative  Ranking  by  PM^S 


NWC 

PMTC 
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ADTC 
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YUMA 
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Relative  Ranking  by  VX^l 
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Relative  Ranking  by  VX-5 
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Overall  Relative  Ranking 
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3.  Conclusions:  Overall,  NWC,  China  lake  was 
clearly  the  most  popular  facility.  Even  VX-4  which  is 
stationed  at  Point  Mugu  and  primarily  uses  facilities 
of  PMTC  preferred  NWC  to  PKTC.  If  familiarity  caused 
an  unfavorable  bias  towards  a  particuliar  facility  this 
did  not  reduce  VX-5'5  ranking  of  NWC  below  number  one 
spot.  It  is  of  no  suprise  that  the  Navy  facilities 
rated  highest  since  respondents  were  mostly  Naval 
Officers.  It  may  be  worthwhile  for  facilities  to 
follow-up  on  these  results  and  find  out  what  makes 
other  facilities  more  appealing  to  potential  users. 
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2y.££tion  J82.  Increased  competition  among  TSE  facilities 
would  increase  the  effectiveness  of  services  provided  to 
project  managers. 

1.  Objective:  In  commercial  enterprises  competition 
has  traditionally  benefitted  customers  since  it 
inherently  fosters  more  efficient  and  effective 
services.  In  the  military,  however,  an  increase  in 
duplicative  functions  may  not  be  feasible  due  to 
political  considerations.  This  question  was  designed 
to  survey  the  opinions  of  users  considering  both  the 
benefits  derived  from  increased  competition  as  well  as 
their  feelings  about  such  a  politically  unpopular 
proposal. 

2.  Results: 


PM'S  20 
VX'S  20 
ALL      40 


PM'S 
VX'S 
ALL 

3»  Conclusions:  In  general,  respondents  did  not 
favor  increased  competition  as  a  viable  means  of 
increasing  effectiveness  of  services.  If  a  majority 
disagreement  had  been  anticipated  it  would  have  been 
worthwhile  to  ask  "why  not  ?".  It  may  be  that 
respondents  consider  outright  competition  in  the 
military  as  an  infeasible  concept. 

£u£3tion  J9:  Competition  among  TSE  facilities  should  be 
eliminated  in  order  to  reduce  the  additional  cost  of 
overlapping  capabilities. 
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1.  Objective:  Related  to  the  previous  question  this 
one  was  designed  to  solicit  opinions  for  total 
elimination  of  competition  due  to  cost  considerations. 
Hopefully  the  respondent  would  weigh  the  benefits 
derived  from  competition  as  it  presently  exists  against 
the  increased  cost  of  overlapping  capabilities. 

2.  Results: 


Responses 
PM'S      20 
VX'S     20 
ALL      40 


5.  A. 
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HE4N 
2.65 
2.7 
2.7 


MEDIAN 
2.214 
2.389 
2.313 


3.  Conclusions:  Overall,  respondents  opposed  to 
increased  competition  were  in  favor  of  reduced 
competition.  The  results  may  have  been  affected  by 
present  public  concern  over  wasteful  military  spending. 
It  is  questionable  whether  or  not  elimination  of 
competition  would  be  financially  beneficial  in  the  long 
run. 

Question   2J?:   Selection   of   a   contractor   influences   the 
selection  of  a  T&E  facility. 

Objective:  Basically  self  explanatory,  this  question 
was  designed  primarily  to  determine  if  a  contractor's 
geographical  accessibility  to  a  facility  or  preference 
to  have  a  particuliar  facility  perform  the  T&E  has  any 
influence  over  the  actual  selection. 
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2.  Results: 

Responses  S.A. 

PM»S     21  0 

VX'S     20  10.0% 

ALL      41  4.93 


PM'S 
VX'S 
ALL 


MEAN 
3.714 
2.950 
3.341 
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14.3% 
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20.0% 
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MEDIAN 

3.875 
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Conclusions:  Although  most  respondents  had  no 
opinion  or  disagreed  it  is  interesting  that  at  least 
some  respondents  felt  contractors  do  have  an  influence. 
It  may  be  worthwhile  for  facilities  to  determine  in 
what  situations  or  for  which  contractors,  contractor 
selection  has  an  influence. 


Question  Zll  DCPs  are  overly  restrictive  in  their  testing 
requirements  and  thus  limit  the  Project  Manager's  selection 
to  one  T&E  facility. 

1.  Objective:  This  question  had  a  two-fold 
objective.  One  was  to  determine  the  degree  of 
confinement  placed  on  the  PM  Ly  the  DC?  or  if  he  was 
permitted  to  utilize  various  facilities  and  make  an 
effective  managerial  decision.  If  the  DCP  were  to 
clearly  dictate  one  T&E  facility,  other  factors  would 
become  unimportant.  Additionally,  it  was  hoped  to  gain 
some  more  insight  as  to  the  uniqueness  of  the  missions 
of  each  facility  and  degree  of  overlap.  It  would  also 
help  to  isolate  the  point  of  time  in  the  planning  c^cle 
when  a  facility  seemed  destined  to  get  the  project. 


2.  Results: 
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Conclusions:  Although  the  question  received  a 
large  number  of  "  no  opinion  ",  there  appears  to  be 
general  agreement  that  most  DCPs  do  not  dictate  the  use 
of  one  particular  facility.  The  large  non-committal 
population  may  indicate  that  DCPs  are  restrictive 
enough  to  indicate  the  use  of  a  particular  facility  but 
nebulous  enough  that  an  aggressive  PM  could  easily 
justify  the  use  of  other  facilities.  It  appears  that  in 
regards  to  satisfying  the  requirements  of  a  DCP  that 
there  is  considerable  overlap  in  missions  of  the 
various  facilities  as  interpretted  by  the  user. 

£u.£§!iL2.Ii  i.2i_  Weapon  systems  T&E  requirements  are  identified 
early  enough  to  permit  funding  justification  for  facility 
modernization. 

1.  Objective;  With  the  advent  of  the  NIF  and  KKTFB 
funding  system,  T6E  facilities  will  undoubtedly  becor 3 
more  cost-conscious  in  the  selection  of  internal 
improvements,  i.e.  only  those  which  will  earn  them  an 
effective  rate  of  return.  The  resulting  capability 
uniqueness  may  require  a  greater  time  period  in  which 
to  prepare  for  the  TCE  requirements  of  a  project.  It  is 
therefore  imperative  that  these  requirements  be 
identified  early  in  the  planning  cycle  so  as  not  to 
cause   delays   later,  on.  Although  it  must  be  recognized 
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that  due  to  the  relative  short  period  of  time  since  the 
introduction  of  NIF  into  T&E  facilities  that  this 
uniqueness  may  just  be  developing,  it  was  desired  to 
get  a  current  basis  for  the  lead-time  problem. 

2.  Results: 


es   S .  A . 

A. 
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0 

26.8% 

22.0% 

34.1* 

17.1% 
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3.  190 

3.333 

3.650 

3.667 

3.415 

3.536 

• 

- 

PM'S  21 
VX'S  20 
ALL      41 

PM»S 
VX'S 
ALL 

3.  Conclusions:  Although  the  PMs  appear  pretty 
evenly  split  over  this  issue,  the  VX  Project  Officers 
appear  to  be  currently  experiencing  some .problems  in 
this  area.  T&E  facilities  may  be  more  in  tune  to  the 
DT&E  aspects  of  programs  than  to  the  OT&E  requirements 
of  OPTEVFOR.  Additionally,  personal  interviews  with  VX 
Project  Officers  indicated  that  schedule  slippage 
during  DT&E  often  caused  a  compression  of  time  allotted 
for  OT&E. 

Question  2.1k   T&s  facilities  are  consistent  in   their   method 
of  assigning  project  priorities  for  completion. 

1.  Objective:  Although  OPNAV  instructions  provide 
general  guidance  for  the  assignment  of  project 
priorities,  it  was  desired  to  ascertain  such  things  as 
if  perhaps  small  projects  received  less  priority  than 
large  continuing  projects  as  a  result  of  the  financial 
impact  on  the  facility.  It  was  also  desired  to  know  if 
a  project  could  expect  equal  treatment  at  several 
facilities  based  solely  on  the  merits  of  the  project. 


97 


2-  Results 


Respon 

ses 

;    All 

Host 

Some 

Few 

None 

PM'S 

17 

0 

29.4% 

23.5% 

41.2% 

5.9% 

VX'S 

19 

0 

42.  1% 

31.6% 

15.8% 

10.5% 

ALL 

36 

0 

36.  1% 

27.8% 

27.8% 

8.3% 

MEAN 

MEDIAN 

PM'S 

3. 

235 

3.375 

\ 

VX'S 

2. 

947 

2.750 

ALL 

3. 

083 

3.000 

3.  Conclusions:  VX  Project  Officers  appear  to 
receive  more  consistent (good  or  bad)  treatment  as 
regards  their  project  priorities.  Project  Managers  tend 
to  disagree  that  there  is  a  consistent  method  of 
assigning  priorities.  Perhaps  one  factor  in  this  is  the 
existence  of  long-term  working  relationships  between 
various  PMs  and  specific  facilities. 

Question  .iiil  TSE  facilities  show  a  genuine  interest  in 
providing  program  support  for  successful  completion  of  the 
project. 

1.  Objective:  This  question  was  designed  to 
determine  the  type  of  environment  a  PM  finds  himself  in 
after  contracting  with  a  facility.  Do  they  follow 
through  with  services  promised?  Are  they  interested  in 
his  project  problems  and  thus  provide  the  "extras" 
beyond  the  requested  tests?  Are  facilities  more 
interested  in  acquiring  new  projects  than  completing 
the  ones  thev  have? 
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Most 

Some 

Few 

None 

42.  1% 

26.3% 

5.3% 

0 

47.4% 

21.1% 

15.8% 

0 

4  4.7% 

Z.  i  •  in 

10.5% 

0 

MEDIAN 

2.063 

2.222 

2.147 

2.  Results: 

Responses  All 

PM»s     19  26.3% 

VX»S     19  15.8% 

ALL      38  21.1% 

MEAN 
PM'S  2.105 

VX'S  2.368 

ALL  2.237 

3.  Conclusions:  Although  the  term  .  "program 
support"  may  not  have  been  adequately  defined,  most 
respondents  seem  pleased  with  the  cooperation  they 
receive  with  their  projects.  VX  Project  Officers  may 
not  receive  the  same  degree  of  support  in  that  their 
projects  are  often  grouped  together  under  the 
cognizance  of  one  program  manager  at  a  facility  due  to 
their  limited  requirements. 

2^.^§^i;2E  25l  Deficiencies  noted  by  the   PM   in   a   project's 
progress  are  examined  and  corrected  by  the  T&E  facilities. 

1.  Objective:  This  was  designed  to  indicate  the 
degree  of  flexioility  of  a  facility  and  also  its 
willingness  to  accept  constructive  criticise  by  the 
user.  It  was  felt  that  the  user  should  be  assured  tnat 
his  viewpoint  would  be  listened  to  and  not  dwarfed  by 
the  magnitude  of  the  T&E  business.  Additionally, 
facilities  should  have  a  system  by  which  these 
deficiencies  may  be  reviewed  and  erradicated. 

2.  Results: 
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PM'S 

20 

10.0% 

4  5.0  % 

20.0? 

20.0% 

5.0% 

VX«  S 

18 

0 

J  j  .  ->  ',i> 

27.8% 

38.9  5 

0 

ALL 

38 
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28.9% 

2  .  O  n 
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MEAN  MEDIAN 

PM«S  2.650  2.389 

VX'S  3.056  3.100 

ALL  2.842  2.722 

3.  Conclusions:  The  results  somewhat  parallel  that 
for  program  support.  VX  Project  Officers,  expectedly, 
do  not  feel  they  have  the  power  to  cause  a  change  in 
deficient  areas  at  many  facilities.  PMs  adopting  a 
more  strongly-opinionated (all  or  none)  stand  than  VX 
Project  Officers  on  this  issue  is  an  interesting 
reversal  to  the  normal  trend  noted  in  other  questions.  ' 

Question  26^  T&E  facilities  capable   of   providing   services 
for  my  project  have  shown  eagerness  for  my  project. 

1.  Objective:  Eagerness  for  a  project  may  be  shown 
in  a  number  of  ways.  Two  noticeable  ways  are  through 
liaison  and  scheduling.  Reference  (31)  indicated  that 
the  "marketing"  effort  of  the  T&E  facilities  varied 
considerably.  NWC  was  cited  as  having  a  very 
well-organized,  well-run  liaison  group  in  the 
Washington  area.  It  was  desired  to  ascertain  how  many 
of  the  facilities  were  able  to  impress  the  managers  of 
their  interests  in  obtaining  projects  through  their 
liaisons.  Eagerness  for  involvement  may  also  be 
detected  by  the  ease  in  which  PMs  can  coordinate  the 
scheduling  of  services  with  a  facility. 

2.  Results: 

Responses  Ail 

PM'S     20  3  5.0% 

VX'S     19  10.5% 

ALL      39  2  3. 1 ^ 


MOSt 

Some 

Few 

None 

3  0.0% 

25.  0^ 

10.0% 

0 

4  2 .  1  ;t 

2&.  3% 

10.5% 

10.5% 

35.9% 

25.6% 

10.3% 

5.1* 

100 


MEAN  MEDIAN 

PM'S  2.100  2.000 

VX'S  2.684  2.438 

ALL  2.385  2.250 

3-  Conclusions:  Due  to  the  relative  difference  in 
dollar  value,  it  is  probably  not  surprising  that  the 
PMs  experienced  a  greater  eagerness  from  the  facilities 
than  did  the  Project  Officers  of  the  VX  squadrons.  What 
is  particularly  noteworthy  is  that  no  PM  answered 
"none",  which  seems  to  indicate  that  facilities  are  not 
saturated  with  projects  to  the  point  of  not  wanting  to 
compete  for  new  projects. 

Question   27^  T&E   facilities   actively   participate  in  the 
selection  process  of  weapon  system  test  assignment. 

1.  Objective:  This  question  was  designed  to 
measure  the  current  extent  of  "  marketing  "  on  the  part 
of  T&E  facilities.  Although  some  of  the  data  on  the 
capabilities  of  the  facilities  are  tabulated  in  the 
various  references  cited  earlier,  it  was  recognized 
that  many  tests  require  very  unique  utilization  of 
equipment.  Since  PM.  staffs  are  limited  in  size  it 
appeared  that,  additional  aid  must  come  from  both  within 
NAVAIR  and  the  facilities.  T&E  facilities  themselves 
seem  most  highly  qualified  to  tell  the  managers  what 
they  can  do  for  them. 

2.  Results: 

Responses  All 

P.VS     18  16. 7 7 

VX'S     18  0 

ALL      36  8.3% 


Most 

Some 
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None 

11.1% 

44.  4% 
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27.  Q% 

27.8% 

33.3% 

1 1 . 1  % 
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13.  9% 
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PM'S 
VX.'S 
ALL 


MEAN 
3.000 
3.278 
3.139 


MEDIAN 
3.000 
3.300 
3.  115 


3.  Conclusions:  Perhaps  the  term  "actively 
participate"  required  better  definition.  The  results 
seem  to  indicate,  however,  that  there  is  not  a  lot  of 
current  involvement  on  the  part  of  the  facility.  While 
this  may  be  good  from  the  standpoint  of  obtaining  an 
independent  decision,  it  seems  that  an  important  source 
of  information  remains  untapped. 

Question  28^  Khat  is  youu  rank/GS  level? 

1.  Objective:  The  objective  was  fairly 
straightforward.  It  was  desired  to  ascertain  this 
information  in  order  to  have  the  ability  to  correlate 
data  from  any  gustion  by  groups  according  to  rank..  This 
also  served  as  a  check  to  see  if  the  questionnaire .  had* 
been  filled  out  by  the  decision  maker  and  thus 
reflected  his  opinions  or  delegated  to  some  other 
individual. 


08   GS9   GS14   GS15 
11      12 

1  1      2 

3.  Conclusions:  It  appeared  that  most 
questionnaires  were  completed  by  the  decision  makeu  cr 
by  high  level  staff  assistants  and  thus  it  was  assumed 
that  they  accurately  reflected  the  thought  process 
behind  the  selection  process. 
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Question  29^  Highest  lev(;l  formal  education  achieved. 

1.  Objective:  As  in  the  previous  question 
additional  background  information  was  desired.  Coupled 
with  question  number  30,  it  provides  valuable  insight 
on  the  backgrounds  of  these  very  important  decision 
makers. 

2.  Results: 

RESP  AA  BS/BA  MS  (TECH)  MS  (NON-TECH)  DOC   DUAL/MS 

PM«S   20  0  25.0%    50. Of*      15.0%  5.0%    5.0% 

VX'S   19  0  7  3.7%    15.8%       5.3%  5.3%     0 

ALL    39  0  48.7%    33.3%      10.3%  5.1%    2.6% 

3.  Conclusions:  As  might  be  expected,  most  PMs 
held  a  Masters  Degree  in  a  technical  area  and  their 
more  junior  counterparts  in  VX  squadrons  held  BS/BA 
Degrees.  All  respondents  held  the  equivalent  of  a 
Baccalaureate  or  higher.  It  is  noteworthy  to  observe 
that  few  respondents  held  Masters  degree  in 
non-technical  areas  even  though  a  large  percentage  of 
their  effort  may  be  in  management. 

Question  30j_  Number  of  years  involved  in  T&E  assignments. 

1.  Objective:  As  in  question  28  and  29  additional 
background  information  was  desired. 

2.  Results: 
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3.  Conclusions:  Question  7  indicated  a  high 
reliance  by  PMs  or.  their  previous  experience  and 
assignments.  The  results  of  this  question  does  not, 
however,  indicate  an  abundance  of  previous  experience 
in  TSE  assignments.  On  the  contrary,  it  seems  to 
indicate  a  definite  need  for  a  strong  liaison  between 
the  "  experts  in  T&E  "  and  the  PM  who  in  many  respects 
is  acting  as  a  corporate  manager. 

C.  EESULTS  AND  CONCLUSIONS 

The  response  received  from  the  questionnaire  was 
excellent.  Preliffiinary  expectations,  based  on  opinions  of 
marketing  experts,  forecast  a  completed  return  of  30-50%  of 
all  questionnaires  sent  out  with  possible  increases 
obtainable  from  additional  follow-up.  The  survey  producer, 
however,  an  overall  response  of  75%.  This  tremendous 
response  seems  to  indicate  a  high  degree  of  interest  and 
concern  in  the  areas  covered  by  the  questionnaire.  It  is 
also  noteworthy  that,  the  questionnaires  appeared  to  have 
been  thoughtfully  filled  out  by  the  decision  makers, 
themselves,  as  indicated  by  the  rank  profile. 

Questions  3  and  22  failed  to  clearly  establish  the 
degree  and  type  of  urgency  experienced  at  the  VX  level. 
Although  the  Project  Officers  did  not  feel  United  to  one 
facility  as  a  result  of  the  urgency  of  their  project,  most 
agreed  that  the  urgency  prevented  adequate  time  for  facility 
modernization  and  improvements  to  complete  the  T&E. 

Questions  1,2,3,  and  21  attempted  to  ascertain  if  and 
why  PMs  might  be  limited  to  one  facility.  Over  60%  of  all 
respondents  felt  that  more  than  one  facility  was  able  to 
provide  the  services  they  required.  Of  those  constrained  to 
one  facility,  technical  constraints  was  the  major  cause.   At 
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the  PM  level,  urgency  contributed  substantially  to  the 
remainder  with  limitations  of  an  overly-restrictive  DCP 
being  relatively  minor. 

Questions  1,  18,  19,  and  26  all  addressed  the  area  of 
competition.  Results  indicate  that  a  competitive 
environment  does  exist  and  installations  appear  to  be  eager 
for  more  business.  Competition  among  facilities  as  viewed  by 
the  Project  Managers  and  Project  Officers  is  not  favored  and 
should  be  reduced.  Futhermore,  it  appears  the  managers 
believe  that  by  not  only  reducing  but  by  eliminating 
competition,  i.e.,  do  away  with  installations1  overlapping 
capabilities,  costs  of  TSE  could  be  lowered. 

Results  of  question  17  indicate  that  Project 
Managers/Officers  perceive  a  difference  in  th^  effectiveness 
of"  various  TSE  facilities.  Results  of  personal  interviews 
by  Pare  personnel  contained  in  Reference.  31  indicate  that 
NWC,  China  Lake,  had  a  better  liaison  group  and  were  easier 
to  work  with  during  project  assignments.  Question  17 
substantiated  these  opinions  since  NHC  enjoyed  a  position  of 
number  1  in  the  eyes  of  the  users.  Perhaps  the  key  for 
facilities  to  improve  Project  Managers'  perception  of  their 
effectiveness  would  be  to  exert  more  effort  towards 
formulating  a  closer  working  relatioship  with  Project 
Managers . 

Responses  to  the  three  cost  guestions,  9,  TO,  and  16, 
appear  to  indicate  some  significant  facts.  Cost  estimates 
are  not  the  number  one  factor  in  importance  for  selecting  a 
facility.  Most  managers  are  aware  of  the  masking  effect 
different  funding  techniques  have  on  total  costs  incurred. 
The  disturbing  feeling  caused  by  a  38,%  no  opinion  response 
on  question  9  might  just  be  explained  by  the  results  given 
in  questions  10  and  16.  Managers  in  1b  appear  to  express  a 
lack   of   crcditability   in   cost   estimates  proposed  by  the 
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facilities  for  T&E  projects.  This  attitude  coupled  with  the 
clouding  of  true  costs  by  the  different  accounting 
techniques  might  generate  within  managers  a  feeling  of 
confusion  and  disinterest  towards  the  fiscal  area. 

Question  16  addressed  the  area  of  facilities*  selection 
criteria.  Ls  mentioned,  cost  considerations  did  not  receive 
the  importance  anticipated.  Technical  capability  was 
clearly  considered  the  most  important  parameter  in  the 
selection  process.  Question  13  pointed  out  that  75%  of  the 
Projsct  Managers  felt  there  existed  a  significant  difference 
in  the  technical  expertise  of  TSE  facilities  offering 
comparable  services.  The  next  section  will  address  this 
area  and  propose  a  method  of  objectively  rating  the  various 
facilities  based  on  their  technical  capabilities. 
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VI.  EFFECTIVENESS  ANALYSIS 

Results  of  the  questionnaire  indicate  that  for  many 
weapon  systems:  more  than  one  TSE  facility  was  capable  of 
fulfilling  the  TSE  requirements  for  a  particular  project. 
Results  also  indicated  that  technical  capability  was  trie 
most  important  parameter  in  selection  of  a  facility  to 
perform  a  weapon  system's  TSE.  There  now  exists,  within 
DoD,  26  major  TSE  facilities,  many  of  which  are  capable  of 
fulfilling  the  TSE  requirements  of  a  specific  project.  To 
the  best  of  the  authors'  knowledge,  no  method  presently 
exists  to  objectively  appraise  the  technical  suitability  of 
these  facilities  for  a  particular  project. 

This  chapter  will  address  the  question  of  which  TSE 
facility  is  most  technically  capable  of  fulfilling  the  TSE 
requirements  of  a  particular  project.  Solution  to  the 
question  is  addressed  through  use  ■  of  a  systems  analysis 
format  (statement  of  objective,  assumptions,  and  measure  of 
effectiveness) .  Through  a  systems  analysis  approach  this 
chapter  demonstrates  an  objective  method  of  selecting  tae 
appropriate  facility  for  a  particular  weapon  scenario.  In 
particular,  this  chapter  presents  a  functional  capabilities 
measurement  of  those  TSE  facilities  capable  ox  providing  TSS 
to  projects  involved  in  aircraft/weapon  systems. 

The  functional  capabilicies  of  the  selected  facilities 
is  quantified  based  on  an.  appraisal  of  qualitative  data. 
The  data  was  based  on  tha  results  of  recent  trios  conducted 
by  PMTC  personnel  to  appraise  the  technical  capabilities  of 
other  TSE  facilities.  It  roust  be  emphasized  that  cost 
considerations  are  ignored  and  that  technical  capabilities 
are  the  prime  considerations  in  the  analysis. 

A.  OBJECTIVE 
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The  objective  of  this  analysis  is  as  follows: 

"Selection  of  the  facility  most  capable  of 
satisfying  the  TSE  requirements  of  an 
aircraft/missile  weapon  system  in  a  capability 
restrained  environment." 

1 •    Lislt at  ions 

tfarely,  if  ever,  does  one  facility  possess  all  the 
equipment/technical  capability  required  to  meet  the  needs  of 
a  specific  project.  This  paper  limits  its  analysis  to 
selection  of  the  facility  which  can  perform  the  majority  of 
the  services  based  upon  current  resources.  ■  In  order  to 
overcome  this  limitation  a  project  manager  has  two  major 
alternatives: 

a)  Additional  funding  for  facility  capability 
expansion. 

b)  Selection  of  secondary  facilities  capable  of 
providing  the  specific  tests  not  available  at  the  primary 
facility. 

The  restriction  of  the  weapon  system  to 
aircraft/missile  is  necessitated  by  the  limitations  of  data 
available.  Point  Mugu,  whose  primary  mission  is  the  T&E  of 
airborne  weapon  systems,  is  the  prime  scource  of  data. 

B.  ASSUMPTIONS 

1  •    One   T&E  facility,  is  the  "P_rirce  contractor^ 

As  mentioned  previously  no  one  facility  has  all  the 
resources  available  to  meet  the  total  requirements  of  a 
project.   Normally   one  (load)  facility  handles  the  majority 
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of  the  T&E  requirements  and  farms  out  to  "subsidiary" 
facilities  specific  tasks  under  the  cognizance  of  the 
Project  Manager. 

2.  More  than  one  facility  can  satisfy  the  T&E 
££3uir_effient  s  °£  §.  BLQimitz. 

Although  the  upper  echelon  executives  of  DoD  may  claim 
that  there  is  little  duplication  of  capabilities  within  the 
Department  this  is  not  so.  It  cannot  be  denied  that  some 
weapon  systems  are  so  unique  that  the  T&E  requirements  can 
only  be  met  by  one  facility.  It  is  not  these  particular 
systems  to  which  this  study  is  directed.  The  survey 
conducted  by  the  authors  indicates  that  many  Project 
Managers  feel  that  more  than  one  facility  was  capable  of 
meeting  the  T&E  requirements  of  his  project.  The  survey  was 
used  to  make  an  assessment  of  the  T&E  process  currently 
employed  by  the  Project  Manager.  A  copy  of  the  survey  is 
included  as  Appendix  B.  Results  are  analyzed  in  Chapter  Y. 
Section  B. 

3.  EE2J£ct  Manager  l_s  selection  not  subject  to 
H2Bzf ilBctional  constraints^ 

This  analysis  presupposes  that  a  Project  Manager's 
selection  criterion  is  based  solely  on  functional 
capabilities.  Other  overriding  considerations  may  limit  a 
Project  Manager's  selection  to  a  less  functionally  suitable 
facility.  Cost  limitations  may  rostict  selection  to  a 
facility  which  can  provide  only  marginal  service.  The 
optimum  facility  may  not  be  able  to  accept  another  project 
due  to  a  backlog  of  work  assignments.  Political  pressure 
beyond  a  Project  Manager's  control  may  force  selection  of  a 
suboptiaial  facility  including  pressure  from  higher 
authority.  A  Project  Manager  may  copsider  additional 
capabilties  more  important  in  the  selection. 
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**•  P£.2JS£i  11§J1^3§£  IS  motivated  to  find  the  optimal 
facility,  for  his  T£E  needs.. 

It  is  assumed  that  the  Project  Manager  is  willing  to 
use  an  objective  criterion  in  selecting  a  facility  to 
satisfy  his  future  T&E  requirements.  The  survey  conductor 
indicates  a  high  dependence  on  familiarity  of  facilities 
from  previous  assignments  as  a  selection  criterion. 
Personal  interviews  conducted  with  Project  Officers  show 
that  some  Project  Officers  felt  quite  adament  about  using 
the  same  T5E  facility  they've  used  in  the  past  and  no  matter 
what  the  capabilities  of  other  facilities  are  they  would 
continue  to  use  the  same  facility  in  the  future.  Here  it  is 
recognized  that  preconceived  notions  have  an  affect  on 
making  an  objective  decision  and  that  a  certain  degree  of 
bias  cannot  be  avoided. 

5«   Project  within  the  cognizance  of   NAVMAT 

Analysis  will  be  limited  to  those  projects  identified 
strictly  as  Navy  projects  under  the  purview  of  the  Navy 
Material  Command  (  NAVMAT) ,  This  avoids  analysis  of  joint 
projects  which  are  concurrently  developed  by  more  than  one 
service.  The  authors'  unf amiiiarity  with  Air  Force/Army 
procurement  particulars  preclude  their  inclusion. 


6. 


Costs  are  not  considered  in  selection. 


U 


Although  it  would  be  unreasonable  to  assume  that  costs 
should  be  ignored  in  the  selection  process  the  survey. 
indicates  that  technical  capability  significantly  outweighs 
considerations  of  cost  estimates  in  the  selection  decision. 
Project  Managers  also  agree  that  the  different  funding 
techniques  used  in  T6E  funding  tend  to  mask  the  actual  costs 
of  a  project. 
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Since  a  cost  analysis  is  not  feasible  at  this  time  it 
must  be  recognized  that  the  computer  program  does  not 
utilize  cost  as  a  parameter*  In  particular,  funding  at  T&E 
facilities  is  in  a  transitional  stage  due  to  the  recent 
enactment  of  the  Uniform  Funding  Policy  (1974)  and  the 
instituting  of  RIF  at  these  activities. 


7. 


Paraochialism  should  not  be  a  selection   criterion 


The  feeling  that  Navy  projects  should  go  to  Navy 
facilities  should  not  be  allowed  to  creep  into  an  objective 
selection  criterion.  Although  facilities  may  have  priority 
methods  that  favor  their  individual  Service  this 
consideration  should  be  dealt  with  after  an  objective 
decision  has  been  made. 

C.  DEVELOPMENT  OF  MODEL 

In  order  to  provide  a  basis  for  relating  the  many 
aspects  of  capability  assessment  it  is  necessary  to 
establish  some  common  denominator.  A  common  denominator 
which  satisfies  the  requirement  for  this  application  is  a 
set  of  functional  capabilities.  Investigation  into  PMTC 
tasks  and  interviews  conducted  led  to  a  determination  of  the 
set  of  26  mutually  exclusive  members  as  developed  in 
Reference  32.  This  set  is  intended  to  exhaust  the 
capabilities  of  a  Test  &  Evaluation  activity,  including  an 
orgranization  as  large  as  PMTC,  in  terms  of 
functions/services. 

The  26  functions  are: 


1.  Trajectory-  Provide  precision  position   measurements 
in   space,  air,  surface,  and  underwater  by  active  or  passive 
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electromagnetic  means,  optical  means,  or  acoustical  means. 
This  includes  radar,  optics,  underwater  tracking,  missile 
impact  location  systems,  LORAC,  and  passive  tracking 
systems: 

2.  Telemetry-  Provide  for  the  collection  and  display  of 
engineering  parameters  such  as  onboard  inertial  data, 
environmental  data,  scalar  miss  distance,  and  other 
performance  data  by  passive  electromagnetic  means. 

3.  Command  Control-  Provide  command  control,  including 
flight  termination  control  of  vehicles  and  missiles,  and  the 
remote  control  of  targets  and  unmanned  vehicles. 

**•  Geophysics-  Provides  environmental  data,  including 
geodetic  surveys  and  meteorological  and  oceanographic 
measurements. 

5-  Freguency  Management-  Provide  coordination, 
monitoring,  and  engineering  services  for  electromagnetic 
freguencies  within  assigned  areas-  This  function  includes 
command  freguency  management,  operation  freguency  support, 
and  area  frequency  coordination. 

6«  £ii£tography-  Provide  photographic,  television,  and 
technical  services  required  to  support  the  documentation  and 
surveillance  of  events  occurring  at  and  about  the  field 
activity  and  the  test  range  as  well  as  the  instrumentation 
support  of  static  and  dynamic  ground  and  airborne  testing 
and  evaluation  activities. 

?•  Launching-  Provide  launching  of  missiles,  test 
vehicles,  and  targets  and  the  ground  support  including 
ordnance  for  launched  systems. 

8.   Hecovery-   Provide   air,   surface,   and   underwater 
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location  and  retrieval  services. 

9.  QESrational  Com  a;unica  tions-  Provide  necessary 
tele-commprovide  capability  to  conduct  and  control,  and 
display  data  for  all  operations,  unication  systems  for  the 
conduct  of  operations. 

10.  Operational  Control-  This  includes  range 
surveillance  and  clearance,  tactical  data  and  real-time 
display,  timing,  range  safety,  range  control,  and  range 
scheduling. 

11.  Threat  Simulations-  Provide  support  to  the  weapons 
systsms  acguisiticn  programs  for  developing,  testing^  and 
evaluation  aerial  and  surface  targets  simulating  realistic 
threat  environments.  7 

12.  Technical  Services-  Provide  prototype  fabrication, 
modifications,  repairs,  calibration,  acceptance  tests,  and 
tenant  range  services  and  depot  level  maintenance  to 
existing  equipment  and  provide  open  shop  computer  services 
for  all  authorized  external  reguestors. 

13.  Internal  £ecjinicj.l  Services-  All  effort  internal  to 
the  activity,  but  not  directly  identifiable  with  the  above 
functional  categories.  This  effort  will  be  prorated  to  the 
functions:   trajectory  through  threat  simulations. 

14.  Facilit ies/U til j ties  Support-  Provide  for  the 
maintenance  and  repair  of  roads,  buildings,  grounds,  and 
eguipment,  and  the  provision  of  utility  services  including 
water,  power,  telepnone,  and  waste  collection  and  disposal 
systems. 

15.  Logistics-  Provide  for  supply  support  services  and 
the   operation   and   maintenance   of  systems  which  transport 
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material  and  personnel  by  land,  sea,  or  air.  Provide 
nesessary  engineering,  management  and  technical  services  to 
maintain  and  support  integrated  logistic  support  plans  for 
assigned  equipment. 

16.  Air  Operations-  Provide  for  the  operation  and 
maintenance  of  all-weather  airfields  including  fire  fighting 
and  crash  crew  support,  flight  support  services  including 
air  traffic  control,  fueling,  other  line  maintenance  and  air 
terminal;  and  search  and  rescue. 

17-  Military  Community  Services-  Provide  services  such 
as  senior  officer  present/area  coordinator,  legal,  chaplain, 
dental,  recreation,  family  housing,  bachelor  enlisted 
guarters,  bachelor  officer  quarters,  and  clubs  for  military 
personnel. 

18.  Security-  Provide  for  the  administration  and 
maintenance  of  a  security  program  for  physical  security,  law 
enforcement,  disaster  control,  and  classified  material 
control. 

19.  Non-operational  Communications-  Provide  for  the 
administration,  operation,  and  maintenance  of  long  range  and 
fixed  point-to-point  telecommunications. 

20.  Administration  and  Management-  This  function 
includes  all  Command  Staff  functions  invoJ ving  resource  and 
program  management,  personnel  and  financial  support 
services,  managment  systems  and  data  processing 

21-  Threat  Assessment-  Provide  for  the  acquisition, 
analysis,  and  assessment  of  performance  and  characteristic 
data  on  aerial  target  and  threat  systems  and  the 
determination  of  the  requirements,  capabilities,  and 
applications   of    aerial   targets   systems   for   simulated 


114 


threats.  Support  to  the  aerial  target  user   community   with 

technical   information   concerning  the  translation  of  threat 

parameters  and  weapon  system  TSE  plans  to   specific   target 
requirements. 

22.  Software  Support-  Provide  for  the  management, 
functional,  and  technical  support  of  all  computer  programs, 
independent  of  their  storage  media,  associated  with  airborne 
weapon  systems,  and  related  design,  analysis,  and 
documentation. 

23.  W^iLESD.  Systems  Test  and  Evaluation-  Provide  the 
operational  planning,  preparation  of  eguipment  and 
resources,  and  performance  of  engineering  ground  and 
airbrone  testing  in  static,  dynamic,  integrated,  or 
simulated  modes  to  acquire  pertinent  data  to  be  analyzed  for 
evaluation  of  procedures,  processes,  capabilities,  and 
characteristics  of  weapon  systems.  The  determination  of 
design,  operational,  and  compatibility  limitations  or 
deviations  as  they  effect  the  development  and  production 
decisions  in  the  weapon  systems  acquisition  cycle. 

24.  Production  Support-  Provide  the  aggregate  of 
engineering  assistance  required  by  a  cognizant  field 
activity  in  support  of  an  ongoing  production  program 
involving  equipments  for  which  it  has  been  assigned 
engineering  cognizance. 

25.  Engineering  Services-  Provide  basic  engineering 
services  for  design,  development,  test  and  evaluation,  and 
product  quality  assessment  in  support  of  the  development, 
production,  and  operational  phases  of  weapon  systems.  The 
services  include  engineering  consideration  of  compatibility , 
safety,  reliability,  maintainability,  and  performance  to 
establish  the  value  of  the  weapon  systems. 
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26.  Aircraft  ?liilLl§ll§.I12S  PeEartraent-  Provide 
intermediate  level  maintenance  on  all  aircraft  assigned  to 
the  facility,  contractor  hailed  aircraft,  tenant  activities, 
and  transient  aircraft;  and  to  provide  organizational  level 
maintenance  to  the  activity  aircraft,  and  transient 
aircraft. 

There  are  26  major  TSE/range  facilities  able  to  act  as 
the  lead  activity  in  a  weapon  system  TSE  program.  These  are 
identified  in  DoD  Directive  3200.11  of  June  18,  1974. 

1.  National  Ranges.  Those  major  DoD  ranges  and  test 
facilities  which  are  unique  national  assets  designed  to 
support  requirements  of  major  DoD  programs. 

Nat  ional  Ranges  Services 

White   Sands   fissile   Ranges  USA 

Kwajalein    Missile   Range  USA 

Pacific   Kissile   Test   Center  USN 

National   Parachute   Test   Range  USN 

•  Eastern   Test   Range  USAF 

Space   and   Missile   Test  Center  USAF 

Satellite   Control   Facility  USAF 
Arnold  Engineering  Development  Center     USAF 

2.  DoD  Major  Test  Facility.  Those  other  major  DoD 
test  facilities  which  support,  almost  entirely,  DoD 
requirements. 

Test  Facilities  Services 

Dugway   Proving   Ground  USA 

Arctic    Test   Center  USA 

Tropic   Test   Center  USA 

i 

Yuma    Proving   Ground  USA 

Jefferson   Proving   Ground  USA 

Electronic    Proving   Ground  USA 
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Aberdeen   Proving   Ground  USA 

Atlantic   Underwater   T&E   center  USH 

Naval   Air   Test   Center  USH 

Naval   Air   Propulsion   Test   Center  USN 

Naval   Air   Test   Facility  USN 

Naval    Weapon   Center  USN 

Atlantic   Fleet   Weapon   Range  USN 

Air   Force   Special   Weapons  Center  USAF 

Tactical  Fighter   Weapon   Center  USAF 

Air   Force   Flight   Test  Center  USAF 

Armament  Development   Test   Center  USAF 

Air  Defense  Weapon  Center  USAF 

Not  all  26  facilities  are  capable  of  supporting  similar 
weapon  systems  T&E,  but  are  restrained  to  one  or  more 
dif ferentiable  categories  such  as  air  related  systems, 
ballistic  missiles,  ordnance,  and  climatic  tests* 
Consideration  of  an  activity  for  T&E  work  is  limited  in 
scope  to  those  organizations  with  related  mission 
statements,  technical  capabilities,  and  expertise  in  the 
broad  weapon  systems  classes. 

One  particular  segment  of  activities  renders  T&E 
support  of  some  degreee  towards  aircraft/missile  related 
systems.  It  is  this  general  weapon  system  area  and 
corresponding  T&E  activities  that  forms  the  boundary  of 
alternatives  available  for  this  particular  selection 
analysis.  This  particular  area  of  selection  was  predicated 
on  availability  of  in-formation  from  a  T&E  activity  within 
this  segment,  PHTC.  Of  the  26  major  T&E/range  facilities  13 
of  them  perform  functions  similar  to  PMTC. 

Table  I  is  a  matrix  representing  the  26  functional 
capaoilities  and  the  1<4  similar/competing  activities. 
Exiscing  capability  is  indicated  by  an  .  x  and  a  lack  is 
represented  by  a  blank  space. 
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Severe  data  and  quantitative  measurement  limitations 
are  evident  in  attempting  to  rate  a  possible  26  functional 
capaoilities  at  14  competing  facilities.  It  was  mandatory  to- 
further  reduce  the  number  of  similar  T&E  activities 
available  to  a  Project  Manager  for  selection  to  the  ones 
whose  functions  have  been  examined.  Information  was  obtained 
on  seven  of  the  fourteen  organizations.  The  seven  activities 
are:  PMTC,  Armament  Development  and  Test  Center  (ADTC) , 
Eglin  AFB,  Uaval  Air  Test  Center  (NATC)  ,  Patuxent  River, 
Atlantic  Fleet  Weapon  Range  (AFWR) ,  Roosevelt  Roads,  Yuma 
Proving  Grounds,  Naval  Weapon  Center  (NWC) ,  China  Lake. 

Determination  of  a  quantitative  measuring  technique  for 
evaluating  activities*  functions  was  discovered  to  be 
impractical.  The  amount  of  technical  expertise  and  T&E 
experience  needed  to  understand  the  various  26  functions 
coupled  with  the  lack,  of  a  common  capability  comparison  data 
base  among  facilities  precluded  the  utilization  of  a 
quantitative  measuring  method. 

A  qualitative  rating  scheme  was  substituted.  The  scheme 
was  developed  by  having  individuals  (at  PMTC)  familiar  with 
T&E  and  its  respective  facilities  render  a  judgement, 
subjective  in  nature,  upon  those  activities'  functional 
capabilities  (Ref,  33) .  Credance  in  their  evaluation  is 
centered  upon  two  known  facts.  The  first  element  is  the 
individual's  experience  in  T&E  and  his  present  work  function 
as  a  program  analyst.  The  second  item  is  the  recent 
completion  of  field  trips  to  the  six  identified 
organizations,  excluding  PMTC  by  a  team  from  PMTC. 
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FACILITIES 


CAPABILITIES 

Trajectory 

Telemetry 

Command  Control 

Geophysics 

Frequency  Management 

Photography 

Launching 

Recovery 

Operational  Communications 

Operational  Control 

Threat  Simulation 

Technical  Services 

Internal  Technical  Services 

Facilities/Utilities 

Logistics 

Air  Operations 

Military  Community  Services 

Security 

Non-3 perational  Communi cat ions 

Administrative    &    Management 

Threat    Assessment 

Software    Support 

Weapons    Systems    T&E 

Production    Support 

Engineering    Services 

Aircraft    Maintenance 


PNSAEWANKAAYAT 
MWADSSUAWFFUFF 
T  C  M  T  T  M  T  T  A  W  F  M  S  W 
C  TCRRECJRTACC 

XXXXXXXXXXXX  X 

xxxxxx  xxxxxxx 
xxxxxx  XXXX  X 

XXXXXXXXXXXX 
XXXXXX     X     XXX     X 
XXXXXXXXXXXX     X 
XXXXXXX     XXXX 
XXXXXXXXXXXX 
XXXXXXX  XX  XXX  XX 
XXXXXXXXXXXX     X 
XXXX     XX        XXX 
XXXXXXXX     XXX 
XXX     X        X     X     X 
XXXXXXXXXXXX     X 
XXXXXXXXXXXX     X 
XXXXXX     X     XXX     X 
XXXXXX     X     XXX     X 
XXXXXXXXXXXX     X 
XXXXXXXXXXXX     X 
XXXXXXXX    xxxxx 
XXXX     X     X     X 
XXXXX        XXX        X 
XXXX     XXX     XXX     X 
XXXX     X     X     X     X 
XXXXXX     X  X 

XXXXXX     X     X     X     X 


Table  I.  Facility  Capability  Matrix 
"X"  denotes  existence  of  capability  at  above  facilities. 
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The  team  was  asked  to  apply  a  qualitative  rating  scale 
consisting  of  four  categories;  better,  average,  worse,  or, 
no  capability,  to  each  of  the  functions  present  at  the 
facilities  visited.  Average  was  defined  as  the  typical  level 
of  capability  existing  for  a  specified  function  after 
surveying  all  the  facilities;  due  to  the  subjectivity  of  the 
investigation,  no  one  activity  should  necessarily  represent 
the  average.  Better  and  worse  classifications  were  easily 
perceived  by  the  team  relative  to  the  average.  A  modified 
ordinal  scale  was  selected  as  the  "yardstick"  best  suited  to 
a  subjective  qualitative  rating  technique.  A  cardinal  scale, 
representative  of  a  quantitative  scale  with  well  defined 
intervals,  could  not  be  adapted  to  a  judgemental  rating 
scheme. 

An  arithmetic  method  was  needed  for  the  transformation 
of  a  qualitative  ranking  into  a  numerical  score.  The  ordinal 
scale  easily  adapted  itself  to  a  numerical  scoring  process 
and  values  of  0  to  3  were  assign  to  the  4  classes  in  this 
order:  0  no  capability,  1  worse,  2  average,  and  3  better- 
Transformation  from  a  qualitative  to  numerical  scoring 
permits  the  utilization  of . a  crude  measure  of  effectiveness, 
(MOE> .  A  HOE  is  necessary  for  satisfying  the  requirements 
of  assigning  weighting  factors  to  the  26  functional 
capabilities;  and,  to  reach  a  conclusion  as  to  the  overall 
standings  of  the  alternative  activities. 

Utilization  of  this  MOE  was  permitted  only  after 
recognizing  three  existing  limitations:  its  subjectivity, 
lack  of  a  well  defined  interval  to  determine  how  much  better 
is  better  from  average  and  average  from  worse,  and  a  rater's 
biasing.  Personnel  rendering  their  opinion  on  the  seven 
facilities'  functional  capabilities  are  members  of  PMTC  and 
as  such  may  have  some  presuppose  tendencies  towards  favoring 
PMTC.  The   ordinal   value   points   assigned   gives   only   an 
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opinion  as  to  the  relative  position  of  the  specific 
functions  at  one  activity  to  the  others.  It  does  not 
indicate  the  degree  of  sa tisf ication  of  a  test  requirement 
which  might  be  obtained  at  the  facility. 

Subjective  rating  scores  of  the  various  activities' 
functional  capabilities  by  the  visiting  team  are  listed  in 
table  II.  The  qualitative  judgement  scheme  used  prohinits 
any  futher  computational  analysis  of  the  scores,  excluding 
totalling  the  points,  such  as  determining  a  numerical  mean 
either  by  function  or  overall. 

Actual  assignment  of  value  points  to  the  capability 
matrix  were  by  necessity  qualitative  in  nature.  Within  a 
functional  element  it  is  felt  that  consistency  exists;  i.e., 
relative  difference  in  a  functional  capability  between 
facilities  with  assigned  values  of  2  and  3,  respectively,  is 
approximately  the  same  as  uetween  facilities  with  assigned 
values  of  1  and  2,  respectively.  However  within  an  activity 
it  is  difficult  to  justify  whether  the  assignment  of  a  1  or 
2  in  one  function  is  analogous  to  awarding  a  1  or  2  to 
another  function.  Since  the  availability  of  data  constrains 
the  research  to  this  form  of  information,  an  assumption  is 
made  that  a  well  defined  ratio  exists  between  functional 
capabilities'  assigned  value  points.  Sensitivity  analysis 
may  justify  or  preclude  this  assumption. 
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FACILITIES 


CAPABILITIES 

Trajectory 

Telemetry 

Command  Control 

Geophysics 

Frequency  Management 

Photography 

Launching 

Operational  Communications 

Operational  Control 

Recovery 

Threat  Simulation 

Technical  Services 

Internal  Technical  Services 

Facilities/ Utilities 

Logistics 

Air  Operations 

Military  Community  Services. 

Security 

Non -Operational  Communications 

Administrative  B  Management 

Threat  Assessment 

Software  Support 

Weapons  Systems  TSE 

Production  Support 

Engineering  Services 

Aircraft  Maintenance 
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3 

2 

1 

1 

Tablell.   -   Facility   Functional   Capability   Rating   (3 
highest)  . 
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D.  COMPUTER  PROGRAM 

The  computer  program  (Appendix  C)  is  designed  to 
determine  a  facility's  ability  to  support  a  given  project 
through  a  simple  linear  relationship. 

The  project  is  initially  analyzed  to  determine  the 
degree  of  importance  of  each  of  the  26  functions  in  the 
completion  of  its  T&2.  Based  on  this  analysis,  weighting 
factors  are  assigned  to  each  funtional  category.  These 
factors  are  then  combined  with  the  previously  defined 
capability  matrix  to  form  the  Linear  Summation  Equation.  For 
example: 


EFAC  =    FUNC(i)  x  CFAC  (i) 

where  EFAC  is  the  effectiveness  index  of  a  facility,  FUNC(i) 
is  the  weighting  factor  assigned  to  each  functional  category 
for  the  project,  and  CFAC  (i)  is  the  assessed   capability   of 
the  facility  in  each  of  the  26  categories. 

The  optimal  facility  is  then  the  facility  with  the 
highest  effectiveness  in  lex.  This  index  can,  as  previously 
noted,  be  subjected  to  a  sensitivity  analysis  and  be 
combined  qualitatively  with  other  factors  such  as  cost, 
availability,  etc. 

The  program  also  has  the  capability  to  do  a  sensitivity 
analysis.  The  weighting  factors  for  each  category  may  be 
varied  by  a  constant  ratio,  CAT1,  CAT2,  etc.  or  a  constant, 
VAR1,  VAR2,  etc.  may  be  added  to  each  individual  weighting 
value. 
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E.  SPECIFIC  WEAPON  SCENARIO 


The  weapon  system  to  be  tested  is  an  air-to-air  short- 
range,  dog-fight  type  missile  with  passive  homing 
capabilities  (infra-rod)  and  its  launch  vehicle  an  F-14/VFAX. 
The  missile  should  be  capaole  of  being  launched  effectively 
within  all  points  of  the  aircraft  combat  envelope  against 
multiple  targets.  The  missile  has  a  25  "g"  turn  capability, 
10  miles  maximum  range,  and  ttach  <4  speed.  Targets  should  be 
representative  of  current  Soviet  combat  aircraft  (HIG  21 , 
MIG  23,  KIG  25,  Su  9,  Blinder) . 

1 .    Specific  Requirements 

The  specific  requirements  of  the  test  and 
evaluation  are: 

a.  Base  two  fighter  aircraft,  at  the  test  site  and 
provide  hangar,  fuel,  oil,  ground  support  equipment,  etc. , 
needed. 

b.  Provide  housing,  messing,  transportation  to 
support  pilots,  crew,  and  technicians.  A  total  of  30  people. 

c.  Provide  ordnance  storage,  handling,  missile 
assembly  and  checkout  for  2  0  missiles. 

d.  Provide  threat  simulation  matching  the  targets 
mentioned  above  as  closely  as  possible  including  multiple 
(2-4)  target  presentation. 

e.  Provide  communication  and  command  control  of 
targets  to  oresent  realistic  maneuvering  capabilities  while 
guaranteeing  the  safety  of  launch  vehicle  crew. 

f.  Provide  six  channel  telemetry  for  data 
collection  and  missile  evaluation. 
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g.    Provide   an   airborne   photographic    package 
capable  of  documenting  missile  shots  and  target  maneuvering. 

h.   Provide  limited  technical  services  in  the  area 
of  T&E  support  for  this  weapon  system. 

i.   Provide  data  processing  for  test  results. 

j.    Program   expected   to  be  completed  six  months 
from  arrival  on  base. 

2-    Task  Categories 


In  order  to  ensure  consistency  and  establish  a  basis 
for  sensitivity  analysis,  the  26  functions  were  grouped  into 
4  categories  as  follows: 

a*  mission  Support-  those  functional 
capabilities  required  to  support  a  mission  airborne  weapon 
system  T&E: 

Aircraft  Maintenance 
Air  Operations 
Frequency  Management 
Geophysics 

Operational  Communication 
Operational  Control 
Recovery 

D»  Test  Requirements-  those  functional 
capabilities  directly  related  to  data  collection  and  test 
conductance : 

Command  Control 
Launching 
(3.)   Photography 
(4.)   Telemetry 

Threat  Simulation 
Trajectory 


(1. 
(2. 
(3. 
(4. 
(5. 
(6. 
(7. 


(10 

(2.) 


(5.) 
(6.) 
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c.  Technical  Support-  those  functional 
capabilities  providing  the  technical  "logistics"  necessary 
to  satisfy  the  T&E  requirements: 

(1.)  Engineering  Services 

(2.)  Internal  Technical  Services 

(3.)  Production  Support 

(4.)  Software  Support 

(5.)  Technical  Services 

(6.)  Threat  Assessment 

(7.)  Weapon  System  T5E 

d.  Base  Support-  those  functional  capabilities 
necessary  to  support  routine  military  operations: 

(1.)  Administration  and  Management 

(2.)  Facilities/Utilities 

(3.)  Logistics 

(4.)    Military    Community    Services 

(5.)     Hon-Operational   Communication    - 

<6.)     Security 

3-    Weightin  g  Factors 

Depending  on  the  weapon  scenario  selected,  each  of 
the  functional  capabilities  has  a  different  degree  of 
importance  and,  therefore,  has  a  different  weighting  in 
deteL mining  the  overall  effectiveness  function  for  each 
facility.  Although  subjective,  a  logical  and  consistent 
approach  to  assigning  weighting  factors  must  be  selected.  A 
hierachy  of  importance  was  initially  chosen  as  shown  in 
Table  III.  The  functional  capabilities  were  ordered  from 
least  important  to  most  important.  In  this  instance  both 
Internal  Technical  Services  and  Production  Support  were  not 
required  and  v/ere  assigned  a  weighting  of  zero.  Both 
Frequency  Management  and  Management  and  Administration  were 
considered  next  in  importance.  This  process  was  continued 
down  the  line  until  the  last  and  most  important  functional 
capability.   Telemetry,   was   assigned  the  highest  rating  in 
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the  hierachy. 

This  initial  approach  to  assigning  weighting 
factors'  was  considered  too  restrictive.  Each  functional 
capability  was  assigned  a  discreet  weighting  factor, 
incrementally  rated  one  unit  more  important  than  the 
preceding  functional  capability/capabilities.  Depending  on 
the  decision  maker's  familiarity  with  the  requirements  of 
the  scenario  in  guestion,  he  may  feel  that  more  than  one 
unit  degree  of  importance  should  be  assigned  to  some 
capabilities.  For  this  particular  scenario  it  was  felt  that 
Command  Control  should  be  placed  two  increments  over  the 
preceding  category  and  that  Telemetry  should  be  placed  three 
units  over  Command  Control.  The  results  are  presented  in 
Table  IV.  In  this  particular  case,  the  weighting  factors 
now  range  from  0  to  15.  Other  personnel,  more  qualified  to 
perform  the  weighting  assignment,  might,  feel  comfortable 
with  a  more  expanded  scale.  An  organization  such  as  NAVAIR 
0-6  may  be  able  to  expand  the  scale  significantly  and  thus 
provide  for  a  more  sensitive  analysis  of  the  data. 

To  provide  a  check  on  the  adequacy  of  the 
weighting  factors,  the  summation  of  weighting  factors  within 
each  general  category  should  be  compared.  For  this  scenario 
the  results  are  as  follows: 


Test  Requirement 


60 


Mission  Support 


40 


Technical  Support 


30 


Base  Support 


15 


The  weighting  of  each  general  category  seemed  to 
reflect  its  relative  importance,  so,  for  this  particular 
scenario,   no   reassessment   of   the   task   categories  totil 
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weighting  factor  scores  was  considered  necessary.  It"  it  was 
felt  that  a  reassessment  was  necessary,  an  iterative 
approach  to  reassigning  weighting  factors  would  be 
apprDpriate.  A  later  section  will  address  this  approach  in 
the  sensitivity  analysis. 

It  must  be  emphasized  that  these  totals  are 
directly  related  to  the  scenario  depicted.  A  weapon  system 
which  greatly  pushes  the  state-of-the-art  might,  for 
example,  require  a  much  higher  value  in  Technical  Support 
whereas  a  system  involving  a  great  number  of  people  might 
require  a  higher  total  ic  Base  Support. 

4.    Data  Usage 

Once  the  weighting  factors  were  assigned,  they  were 
placed  into  the  Linear  Summation  Equatipn  from  which  the 
computer  program  outputted  an  ordered  ranking  of  facilities 
by  technical  capabilities.  Depending  on  the  degree  of 
confidence  the  decision  maker  places  on  his  weighting 
assignment,  he  may  want  to  input  another  set  of  weighting 
factors  to  ascertain  if  they  will  affect  the  results.  This 
approach  is  addressed  in  the  next  section,  the  sensitivity 
anal/sis.  If  the  new  set  of  weighting  factors  affects  the 
effectiveness  ordering  of  the  facilities,  a  reassessment  of 
critcal  factors  is  needed.  If  the  order  is  unaffected,  no 
further  assessment  for  determining  facilities'  technical 
capapilties  for  the  scenario  is  needed 


128 


FUNCTIONAL  CAPABILITY  HIERARCHY 

1.  Internal  Technical  Services  0 
Production  Support  0 

2.  Frequency  Management  1 
Adraisistrative  and  Management  1 

3.  Geophysics  2 
Logistics  2 
Non-Operational  Communications  2 
Engineering  Services  2 

4.  Facilities/Utilities  3 
Military  Community  Services  3 
Operational  Conuaunications  3 

5.  Security  4 

6.  Recovery  5 

7.  Technical  Services  6 
Threat  Assessment  6 
Software  Support  6 
Aircraft  Maintenance  6 

8.  Photography  7 
Launching  7 

9.  Air  Operations  8 

10.  Threat  Simulation  9 

11.  Trajectory  10 
Operational  Control  10 
Weapon  Systems  Test  &    Evaluation  10 

12.  Command  Control  11 

13.  Telemetry  12 

Table  III.  Hierarchy  of  Functional  Capabilities 

Rating   of  the  26  functional  capabilities  from  least  to 
most  important  in  numerical  order. 
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FUNCTIONAL  CAPABILITY         WEIGHTING  FACTORS 

1 .  • Tra  jectory  10 

2.  Telemetry  15 

3.  Command  Control               t  12 

4.  Geophysics  2 

5.  Frequency  Management  1 

6.  Photography  7 

7.  Launching  7 

8.  Recovery  5 

9.  Operational  Communications  3 

10.  Operational  Control  10 

11.  Threat  Simulation  9 

12.  Technical  Services  6 

13.  Internal  Technical  Services  0 

14.  Facilities/Utilities  3 

15.  Logistics  2 

16.  Air  Operations  8 

17.  Military  Community  Services  3 
13.    Security  4 

19.  Non-Operational  Communications  2 

20.  Administrative  and  Management  1 

21.  Threat  Assessment         .  6 

22.  Software  Support  6 

23.  Weapons  Systems  Test  S  Evaluation     10 

24.  Production  Support  0 

25.  Engineering  Services  3 

26.  Aircraft  Maintenance  6 

Table  IV.  Rating  of  Functional  Capabilities 

Finalized   weighting   factors  used  in  dog  fight  missile 
scenario. 
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F.  RESULTS  AND  SENSITIVITY  ANALYSIS 

A  computer  solution  (Appendix  C)  was  obtained  for  the 
example  outlined  using  the  stated  values.  In  addition, 
initial  sensitivity  analysis  was  conducted  by  varying  the 
category  totals  in  such  a  manner  as  to  keep  the  total  point 
value  of  145  constant.  In  all  cases  the  Base  Support 
category  was  held  constant  since  it  was  felt  its  assigned 
value  accurately  depicted  its  importance.  Changes  were 
uniformly  made  on  all  elements  within  a  category  by  inputing 
a  ratio  value  into  CAT1  through  CAT4.  Alteration  of  CAT1 
through  CAT4  investigated  the  sensitivity  of  total  task 
categories'  assigned  values. 

The  first  two  cases  show  the  effect  of  first  decreasing 
Test  tfeguirements  by  16  percent  and  then  increasing  it  by 
the  same  amount  and  evenly  distributing  the  difference  to 
Mission  Support  and  Technical  Support. 

The  next  two  cases  depict  the  subtraction  and  then  the 
addition  of  20  percent  (8  points)  on  Mission  Support. 
Constant  point  value  was  maintained  by  evenly  distributing 
the  difference  to  Test  Requirements  and  Technical  Support. 

The  final  two  cases  depict  a  similar  subtraction  and 
addicion  of  20  percent  (6  points)  on  Technical  Services. 

In  all  cases  above  the  relative  ranking  order  remained 
constant,  although  the  absolute  difference  in  scores 
changed.  ADTC  closely  challenged  PMIC  throughout  the 
analysis  for  the  number  one  position.  A  maximum  difference 
in  scores  between  the  two  facilities  remained  within  three 
percent.  The  analysis  shows  that  the  values  placed  on 
category  totals  is  not  highly  critical  to  end  results 
throughout  a  reasonable  range  of  values  (15-20  percent). 
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Weighting  factors  within  a  task  category  were  next 
changed  independently  by  inputing  an  adjustment  vector 
VAR  (i)  .  Throughout  this  analysis  the  total  weighting  factor 
of  each  task  category  was  held  constant.  This  analysis 
assumed  a  properly  assigned  category  total  but  checks  the 
sensitivity  of  assigned  functional  values  within  a  category. 
In  this  example  only  the  values  with  in  the  Test  Requirement 
category  were  altered  but  similar  analysis  could  be 
performed  on  the  other  three  categories. 

First,  5  points  were  subtracted  from  the  Telemetry 
value  and  1  point  was  added  • to  each  of  the  other  five 
functional  areas.  It  should  be  noted  that  this  altered  the 
functional  hierarchy  within  the  category  by  placing 
Telemetry  subordinate  to  Command  Control.  Next,  the 
category  hierarchy  was  held  constant  by  subtracting  1  point 
from  Command  Control  and  3  points  from  Telemetry  and  adding 
1  point  to  each  of  the  other  four  areas.  Finally,  all  areas 
within  the  category  were  uniformly  assigned  a  value  of  10 
points. 

Results  indicate  no  change  in  the  hierarchy  of 
facilities'  totals  and  little  relative  change  between  two 
consecutive  facilities.  In  this  scenario  the  tas«:  category 
total  seems  to  be  the  dominated  factor  and  little 
sensitivity  is  shown  by  manipulating  each  of  the  sir. 
variables  within  the  Test  Requirement  category.  Sone 
facilities  had  the  same  values  assigned  in  their  capability 
vector  in  all  effected  functional  area  and  thus  showed  no 
change  in  effectiveness  total  (eg.  WSMR) . 

Thus  it  appears  that  once  the  faclitiess  are  accurately 
rated  by  functional  capabilities,  a  Project  Manager  may 
assign  his  own  weighting  factors  with  confidence. 
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G.     CONCLUSIONS 

Suitable  facility  selection  for  adequate  testing  of 
weapon  systems  is  an  important  prerequisite  to  acquiring 
combat  functional  weaponry.  The  present  method  of  selection 
has  drawbacks  due  to  the  subjectivity  of  the  selection 
process;  parameters  somewhat  irrelevant  to  proper  selection 
frequently  weigh  heavily  in  the  final  selection.  If  the 
Navy  is  to  improve  its  system  acquisition  process  a  more 
ob-jective  criterion  must  be  implemented  in  the  selection 
process. 

This  analysis  has  demonstated  a  rational  method  of 
objectively  assessing  the  functional  suitability  of  a 
facility  for  fulfilling- the  TSE  requirements  of  a  particular 
project.  Time  constraints  and  data  limitations  restricted 
the  detail  and  objectivity  of  the  analysis.  In  particular, 
the  data  source  was  based  on  the  opinions  of  personnel 
attached  to  the  same  facility.  This  may  inherently  sway 
their  impressions  of  competing  facilities.  The  restrictive  0 
-  3  rating  scale  employed  eliminated  the  possibility  of 
distinguishing  between  capabilities  that  were  just  above 
average  and  those  that  were  totally  superior  tc  tneir 
competitors.  The  sensitivity  analysis  points  out  the 
importance  of  an  expanded  rating  scale  since  weighting 
factor  variations  as  large  as  20  percent  were  ineffective  in 
altering  the  overall  ranking.  It  is  believed  that  if  more 
objectivity  and  an  expanded  rating  scale  were  employed  a 
differentiation  in  ranking  would  have  resulted. 

Follow-on  study  is  recommended  in  this  area  since  the 
analysis  presents  a  straight  forward  method  of  ranking 
facilities  through  a  weighted  linear  summation  of  26 
identifiable  functional  capabilities.  To  reduce  bias, 
personnel  at  the  NAViiAT  or  DoD  level  should  conduct  a 
survey   of   TSE    facilities    and    rank    the    functional 


133 


capabilities  on  a  scale  of  0  -  10.  If  a  facility  expands 
its  functional  capabilities  or  for  some  other  reason  feels  a 
higher  rating  is  warranted  the  facility  could  appeal  for  a 
review  of  its  ranking  in  the  questionable  areas.  This 
ranking  method  could  also  provide  useful  feedback  to 
facilities  for  determining  their  relative  effectiveness  and 
help  identify  areas  needing  improvement.  When  a  Project 
Manager  determines  his  T&E  requirements  a  few  computer  carls 
could  provide  him  with  a  list  of  the  most  qualified 
facilities. 

This  analysis  does  not  pretend  to  propose  the  ultimate 
solution  to  the  decision  maker.  Prior  to  final  selection 
the  Project  Manager  might  have  to  weigh  other  less  tangible 
considerations  or  parameters  not  yet  identified  by  the 
analysis  such  as  costs,  time,  and  scheduling  problems. 
However,  this  type  of  analysis  reduces  the  difficulty  of 
making  the  proper  selection. 
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VII.  CONCLUSIONS  AND  RECOMMENDATIONS 

This  chapter  states  what  was  accomplished  by  the 
research  in  view  of  the  five  objectives  covered  in  the 
introduction.  An  attempt  is  also  made,  where  applicable,  to 
recommend  improvements  and  to  point  out  areas  where 
additional  research  could  be  accomplished. 

A.  WEAPON  SYSTEM  ACQUISITION  PROCESS 

Research  of  current  directives  showed  that  T&E  facility 
missions  were  not  clearly  defined  and  thus  some  overlap 
existed.  NAVMATINST  5450T27  acknowledges  the  existence  of 
overlap  and  encourages  a  limited  amount  of  competition. 

Although  the  Project  Manager  is  granted  a  great  amount 
of  authority  and  responsibility  by  a  Project  Charter,  few 
procedures  seem  to  be  clearly  defined  in  the  T&E  facility 
selection  process.  Current  directives  seem  to  stimulate  a 
buyer-seller  relationship  between  the  PM  and  the  T&E 
facility. 

Previous  attempts,  such  as  the  establishement  of  TECO, 
have  been  unsuccessf ull  in  providing  for  overall 
coordination  of  T&E  within  NAVAIR.  The  establishment  of 
AIR-06  is  an  attempt  to  strengthen  this  coordination,  but 
its  effectiveness  remains  to  be  tested.  There  appears  to  be 
a  problem  of  overlapping  r9sponsibilities  in  the  control  of 
resources  inherent  in  rue  proposed  system  due  to  the 
double-hatting  of  06T  and  06R  as  pointed  out  in  Chapter  III. 

B.  FACILITY  ORGANIZATION 

In  order  for  an  organization  to  be  effective  it  must  b< 
responsive    and    adaptable    to    its    changing   external 
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environment.  This  is  particularly  true  of  T&E  facilities 
which  are  so  intimately  involved  witn  a  rapidly  changing 
technology.  The  stagnation  that  has  hampered  operations  at 
the  Point  Mugu  complex  must  be  set  aside.  The  PMTC 
organization  is  structured  to  avoid  stagnancy  and  to  be 
reactive  to  its  users'  needs.  However,  an  organization  cm 
only  be  as  effective  as  the  people  working  for  it.  Only 
time  can  determine  if  the  personnel  are  willing  to 
participate  in  its  dynamic  development. 

C.  EFFECTS  OF  FUNDING  METHODS 

The  authors  had  the  impression  that  the  primary  driving 
force  behind  a  PM's  selection  of  a  T&E  facility  was  cost. 
Research  into  the  means  by  which  activities  receive  their 
funds  revealed  that  there  are  two  major  financial  management 
techniques  used  by  facilities.  They  are  NI?  and  RMS. 
Additionally,  the  major  T&E  facilities,  as  specified  in 
DODDIR  3200,11,  have  procedures  and  regulations  governing 
their  funding  as  a  result  of  being  MRTF3  members.  In 
general,  then,  a  T&E  facility  is  either  under  NIP  or  RMS  for 
its  financial  management  and  receives  its  funds  either  from 
NIF  or  HRTFB/lnstitutional  Funds. 

NIF  appears  to  £>e  a  more  dynamic  managerial  method  than 
RMS.  NIF  requires  an  activity  to  receive  its  operating 
funds  through  the  sale  of  goods  and  services  to  customers. 
This  causes  several  economic  consequences:  formation  of  a 
buyer-seller  relationship,  a  need  for  customers,  a  need  to 
ensure  their  satisfaction,  and  the  formulation  of  a 
competitive  environment.  RMS,  in  contrast  to  NIF,  only 
encourages  a  faciltiy  to  efficiently  use  its  resources.  An 
RMS  activity  is  thus  guaranteed  of  a  user  for  its  services 
and  does  not  need  to  seek  customers  or  ensure  their 
satisfaction. 


Ub 


The  difference  in  funding  policy  causes  different 
charges  to  the  user  as  discussed  in  Chapter  IV. 

D.  PROJECT  MANAGERS'  PARTICIPATION 

Respones  from  the  guestionnaire  indicated  that 
technical  capabilities  an  installation  can  offer  for  a 
project  was  the  most  important  criterion  used  by  decision 
makers  in  selecting  a  T&E  facility.  In  contrast  to  the 
authors'  original  assumption,  cost  estimates  were  not  number 
one  but  a  distant  third  after  technical  capability  and 
personnel  expertise. 

A  Project  Manager  has  considerable  latitude  in 
selecting  a  T&E  facility  and  is  not  normally  hampered  by 
external  constraints  such  as  project  urgency  or  higher 
authority  prior  selection.  As  was  expected,  he  could  be 
limited  in  his  selection  process  by  technical  constraints 
to,  possibly,  one  facility;  and,  approximately  45  percent  of 
the  PMs  were.  In  addition  to  the  latitude  a  PM  enjoys 
during  initial  facility  selection,  he  can,  later  on,  alter 
the  selection  to  another  facility. 

The  process  used  by  Project  Managers  to  learn  of  an 
installation's  capability  was  based  heavily  upon  past 
experience  and  associar.es.  Next  in  importance  was  the 
liaison  offices  established  in  Washington.  The  liaisons 
were  used  by  a  majority  of  the  PMs,  but  the  services  could 
be  improved.  Reference  31  indicated  that  some  liaison 
offices  were  better  than  others,  which  would  have  a  direct 
influence  on  a  Project  Manager's  perception  of  that 
facility.  Publications  were  not  considered  a  useful 
information  source  by  the  managers. 

Project   Managers    do    not    desire   a   competitive 
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environment  in  T&E.  They  believe  the  cost  of  T&E  would  be 
reduced  if  competition  among  facilities  were  eliminated. 
This  view  is  in  conflict  with  NAVMATINST  5U50.27  which 
expressed  the  opinion  that  limited  competition  is  both 
necessary  and  good  for  T&E. 

Responses  from  the  questionnaire  indicated  a  clear 
preference  for  NWC  and  the  services  it  offers  over  other 
comparable  facilities.  This  high  degree  of  satisfaction 
with  NWC  was  held  by  both  Project  Managers  and  Project 
Officers.  It  would  be  to  the  benifit  of  other  facilities  to 
investigate  this  finding  further  and  determine  the  reasons 
behind  the  NWC  number  one  ranking. 


E. 


AN  OBJECTIVE  SELECTION  PROCESS 


The  facility  selection  process  developed  in  Chapter  VI 
provides  a  systematic  means  by  which  a  PM  could  determine 
which  facility  is  most  technically  capable  of  handling  his 
project.  The  26  functional  areas  defined  by  PMTC  personnel 
seem  to  exhaust  the  capabilities  of  a  T&E  facility  in  terns 
of  services.  Although  it  is  recognized  that  the  point 
totalization  could  be  hand  calculated,  the  computer  program 
aids  greatly  in  a  sensitivity  analysis  of  weighting  factors. 

There   exists  a  need  to  make  an  objective  determination- 
of  a  facility's  rating  in  each  of  the  26  areas  and  a   method 
of  updating  it.  This  could  oe  effectively  accomplished  by  an 
organization  such  as  AIR-06. 

Additional  research  could  be  done  in  developing  a  more 
objective  and  easily  understandable  method  of  assigning 
project  weighting  factors.  This  might  be  done  by  a  form 
which  asks  specific  questions  concerning  the  number  of 
personnel   to   be   based   incident   to   a   project   etc.  and 
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assigning  point  values  accordingly. 

It.  must   be   emphasized   that   the  process  is  based  on 

technical  capability,  only,  and  does  not  address  itself   to 

other    factors   such   as   cost   which   must   be  eventually 
considered. 
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Appendix  A 

PHR  Financial  Budget  FY  74-81 
(in  million  of  dollars) 

74       75  76       7T       77 

Operations 

Institutional      73.97    68.70  78.92    21.38    85.53 

Direct  Charges      8.0     20.55  19.53    5.19    20.76 

Range  Improvements 

Institutional       5.2     12.29  9.8      3.15    24.85 

Others              3.67     2.54  11.64     1.21     8.41 

TOTALS     90.84   104.08  119.89    30.93   139.55 

78       79  80       81 
Operations 

Institutional      87.33    88.35  88.72    89.46 

Direct  Charges     20.76    20.76  20.76    20.76 
Range  Improvements 

Institutional      23.84    20.10  18.92     5.46 

Others              8.09     4.08  1.08     0.0 

TOTALS    140.51   133.29  129.48   115.68 
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PMR  Direct  Charges  (Users1  Funding) 
(thousands  of  dollars) 


Nav^ 


RDT&E,N 

0&M,N 

NIF 

Oth; 

er  DoD 

RDT&E,AF 

0&M,AF 

0&M,A 

Non 

D£D 

AEC 

NASA 

Non 

Federal 

TOTALS 

17^600 

17,500 

300 

300 

3,300 

3 

,400 

14,000 

13 

,800 

2A300 

900 

1,900 

300 

320 

525 

80 

75 

500 

0 
500 

J50 

20,550 


800 

440 

360 

150 
19,350 


Projected  Total  Operations 


Range  User 


Navy 

NASA 

Civilian 

DoD 

AEC 

USAF 

Array 


CY  75 

14,293 

32 

0 

12 

77 

942 

232 


TOTALS  15,568 


CY  76 

12,847 

32 

0 

12 

76 

896 

172 

14,03  5 


CY  77 

12,537 

32 

0 

12 

76 
885 
172 

13,714 


CY  78 

12,651 
32 

0 

12 

76 

837 

232 

13,840 
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Naval  Missile  Center 

(FY  75) 

Customer  Work  Requests 

(dollar  value) 

NAVAIR  $94,674,383 

NAV3EA  $5,723,584 

NAVELEC  $1,732,526 

CNR  $252,065 

Other  Navy  $16,077,418 

Marines  $211,349 

Air  Force  $3,389,420 

Army  $1,139,714 

NASA  $39,692 

Other  Govt.  Dept.  $294,997 

Private  Parties  $23,630 

ASO  $28,936 

NAVILCO  $4,242,761 

TOTALS  $127,830,475 
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APPENDIX  B 


information  on  the  capabilities  of  T&E  facilities. 

6.  Liaisons  from  T&S  facilities  provide  a  useful  source 

of  information  on  the  capabilities  of  their  facilities. 

7.  A  project  manager's  previous  assignments  provide  a 
valuable  source  of  Information  on  the  capabilities 
of  T&E  facilities. 

8.  Staff  assistance  within  a  system  command  Is  useful 
in  determining  the  capabilities  of  T&E  facilities. 

9.  There  is  a  significant  difference  in  costs  to  projects 
from  T&E  facilities  offering  comparable  services. 

10.  Different  funding  techniques  (KIP/RMS, etc. )  used  by 
T&E  facilities  tend  to  mask  the  actual  T&E  cost  of 
a  project. 


□ 


TEST  A1JD  EVALUATION  ASSESSMENT 

Please  enter  the  most  appropriate  number  in  the  box  at 
the  right  of  each  statement. 
l.=Stronc-ly  Agree  2.=Agree  3«=No  Opinion  *+.=Dl  sagree  5«  =  Strongly  Disagree 

1.  More  than  one  Test  and  Evaluation  (T&E)  facility  was 
capable  of  providing  the  simlllar  services  needed  for 
my  project. 

2.  The  technical  constraints  of  my  project  limited  selection 
to  one  T&E  facility. 

3.  The  urgency  of  my  project  limited  selection  to  one  T&E 
facility. 

**.  Current  publications  provide  a  useful  source  of 

information  en  the  capabilities  of  T&E  facilities. 
5.  Other  project  managers  provide  a  useful  source  of 


□ 
□ 
□ 
□ 

□ 

□ 

D 
□ 

□ 
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Please  enter  the  most  appropriate  number  In  the  box 
at  the  right  of  each  statement. 
l.=Stronp;ly  A?ree  2.  =  A£r_ee  3«=No  Opinion  U.=D1  sap;ree  5»=Strongly  Disagree 

11.  Once  a  T&E  facility  has  been  chosen  for  a  project  a  

project  manager  has  no  latitude  In  altering  the  I | 

selection. 

NOTE:  For  the  following  3  questions  comparable  services  is 

meant  to  be  those  services  which  more  than  one  T&E  facility 
Is  capable  of  providing  in  support  of  a  project. 

12.  As  regards  to  facilities  offering  comparable  services, 
there  is  a  significant  difference  in  the  information 
feedback  provided  to  a  project  manager  to  effectively 
monitor  the  progress  of  his  project. 

13.  There  is  a  significant  difference  in  the  technical  

expert! se  of  T&E  facilities  offering  comparable  services.       | | 

lU.  As  regards  to  facilities  offering  comparable  services,  

there  is  a  significant  difference  in  the  ease  of  obtaining      | | 

non-routine  information  on  the  progress  of  a  project. 

15.  Funding  level  for  T&E  facilities1  modernization  is 
adequate  to  support  my  project  needs. 

16.  Kank  each  of  the  following  factors  with  respect  to  its 

Importance  in  the  facility  selection  process. 

Cost  Estimates 

Personnel  Expertise 

Technical  Capability 

Logistics    .  f — j 

Program  Cooperation  # — . 

Test  Scheduling  |=j 

Others  (specify)  I — ' 


□ 
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Please  enter  the  most  appropriate  number  In  the  box  at  the 
right  of  each  statement. 

l«=Strong;ly  Agree  2.=Agree  3»=No  Opinion  U.=Dlsagree  5»  =  ^tronp;Iy  Disagree 

17.  Rank  the  following  T&E  facilities  In  their  overall 
effectiveness  In  providing  satisfactory  services  to 
project  managers. 

1.  Host  effective  8.  Least  effective 

A.  Naval  Weapon  Center,  China  Lake 

B.  Pacific  Missile  Test  Center,  Pt.  Mugu 

C.  Naval  Air  Test  Center,  Patuxent  River 

D.  Armament  Development  and  Test  Center, 
Eglin  AFB 

E.  White  Sands  Missile  Ranee 

F.  U.S.  Army  Yuma  Proving  Grounds 

G.  Altantlc  Fleet  Weapons  Range 
H.  Air  Force  Eastern  Test  Range,  Patrick  AFB 

18.  Increased  competition  among  T&E  facilities  would 
increase  the  effectiveness  of  services  provided  to 
project  managers. 

19.  Competition  among  T&E  facilities  should  be  eliminated 
in  order  to  reduce  the  additional  cost  of  overlapping 
capabilities. 

20.  Selection  of  a  contractor  Influences  the  selection 
of  a  T&E  facility. 

21.  DCP's  are  overly  restrictive  In  their  testing  requirements 
and  thus  limit  the  project  manager's  selection  to  one 
T&E  facility. 

22.  Weapon  systems  T&E  requirements  are  Identified  early 
enough  to  permit  funding  Justification  for  facility 
modernization. 


1^5 


D 

a 
□ 
a 


a 


a 


a 
a 


When  answering  the  following  questions  use  the  appropriate 
numerical  response  as  Indicated  below. 
1.=  All   2.=  Most   3.=  Some  k.=   Few   5.=  None 

23.  T&E  facilities  are  consistent  In  their  method  of 
assigning  project  priorities  for  completion. 

2k.    T&E  facilities  show  a  genuine  Interest  In  providing 
program  support  for  successful  completion  of  the 
project. 

25.  Deficiencies  noted  by  the  P.M.  in  a  project's 
progress  are  examined  and  corrected  by  the  T&E 
facilities. 

26.  T&E  facilities  caoable  of  providing  services  for  my 
project  have  shown  eagerness  for  my  project. 

27.  T&E  facilities  actively  participate  in  the  selection 
process  of  weapon  system  test  assignment. 

28.  What  is  your  rank/GS  level? 

29*  Highest  level  formal  education  achieved. 

Associate Baccalaureate  Waster ( technical ) 

Master(non-technical )  Doctorate 

30.  Wumber  of  years  involved  in  T&E  assignments. 

31*  Do  you  desire  results  of  this  questionnaire? 

YES        MO 

Mailing  Address  


form  number 


□ 

□ 
□ 

D 
□ 

□ 
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